Advanced Search

Please click here to take a brief survey

Comparative Measurements and Knowing Our Facts
Alex Steffen, 16 Feb 09

304526237_6d1acf58bb.jpgWe all see sustainability comparisons regularly: "...if Americans stopped buying red, round clown noses, they'd save as much energy as it takes to make all the pogo sticks used worldwide."

These are fun. Sometimes these are clever. Unfortunately, these are also almost always completely useless, for the following reasons:

1) Often, we don't know what the folks making the comparison mean by their terms or how they got their numbers, and thus, whether those numbers are reliable. The frequency with which numbers appear to be kind of arbitrary is pretty alarming. Many are essentially just bits of folk wisdom within the environmental movement.

2) Different comparisons often count the same things in different ways (what's known as the boundary condition problem) -- so you get architects claiming that green buildings could save 40% of CO2 and energy geeks claiming that energy efficiency could save 35%, but that doesn't mean that green buildings and energy efficiency together could save 75%, because they're both counting many of the same things in different ways.

3) Even when they're counting different things in the same way, comparisons are confusing. I once considered writing a post about how we were running out of SUVs to take off the road -- because of solutions proposed in 20 or so different stories I had recently read. Every one of those stories used the same comparison ("X would be like taking # of millions of SUVs off the road"), which made them all paradoxically more confusing and blurry... plus how many times can we take the same SUVs off the same roads?

Even for numbers people, which many of us aren't, these things are confusing. But I think there's a real need to identify a common set of absolute measurements (tons of CO2s, watts, gdp, for instance) and a set of agreed-upon ways to describe the boundaries you're assuming when claim certain impacts, so we're all measuring apples against apples.

For instance, in just the last four days, I got two emails with the following assertions: that the operation of buildings counts for 60-80% of cities' greenhouse gas emissions in North America, and that driving cars is responsible for 45% of the average U.S. family's emissions.

Now, there is probably a way to slice each of these assertions so that it is sort of true. We could, for instance, decide not to count any economic activity (from farming to manufacturing) that happens outside a city's boundaries, or measure any of the embedded carbon in its buildings and infrastructure (in other words, just count what we're emitting right now, not what it's taken to build what we have or what we'll build in the future), while counting all activities within those buildings as part of operations and maybe get to a 60-80% share of carbon footprints from building operations. Similarly, we could choose not to measure all sorts of other things but to measure all of the lifetime impacts of owning a car and maybe come up with a 45% share of a family's emissions from driving, but you'd have to really distort the usual picture (like, say, assigning a family's agricultural climate footprint to the farmer who grew their food) to get there.

Both of these numbers came from smart, honest sources, but in too many cases, advocates choose to measure different things in different ways in order to get to a number that supports their preferred climate action, or just see putting these sorts of statistics into the world in purely utilitarian terms: if it gets people to act, why quibble about the details?

I see these things differently. If these numbers are going to mean anything, and be of any use, they need to refer to the same things, or at least the same kind of things. Otherwise we risk distorting our understanding of the problem so much that we will be unable to find actual solutions, or waste time emphasizing solutions that don't actually help much.

That means we need to learn our business with these numbers.

If you have a favorite climate impacts measurement tool, I invite you to share it -- and the argument in its favor -- in the comments below.

Photo credit: flickr/ansik, Creative Commons license.

Bookmark and Share


Agreed - as someone wrote once in a similar context, "I don't care what your plan is, but it's got to add up!"

Sometimes, getting the numbers wrong can really detract from the message, too; if somebody doesn't have enough of a feel for the situation to realise that they're out by four orders of magnitude, for instance, why should I believe the rest of what they say? (And yes, I've really seen that four orders of magnitude, a factor of 9000. Somebody dropped an "hours/year" factor without noticing.)

Measure carefully, do your sums! If you have obviously wrong numbers, it helps nobody and probably harms the cause.

Posted by: sabik on 18 Feb 09

Excellent point! All too often the environmental movement has a reputation among government and business communities for being "fuzzy-headed" for just this kind of thing. It makes a a great talking point if you're only interested in fundraising or getting quoted in the media, but if you're trying to get invited to the table where decisions are made you need hard data in standard units that everyone else at the table can understand and use.

The same issues apply for health effects of environmental problems - if you're claiming a source of pollution is causing a cancer (or other disease) cluster or outbreak, please be aware that a cluster is defined as one specific set of symptoms with a specific single cause - a hard thing to demonstrate, which is why there are only a handful of environmentally-related health clusters across the US accepted by the medical community... Saying one-third of the people in your local church have died of cancer due to the dump next door doesn't define a cluster - one third of people across the US die of cancers, taken as a whole. Sad but true.

Posted by: Knoxville Tennessee on 20 Feb 09

The one that drives me a bit around the bend is "using this whizbang gadget will save 1.2 tonnes of carbon emissions!"

While this may be useful for comparison purposes we should realize there is no 1 to 1 relationship here. Just because I use 1kw less electricity at my end of the wire there is no corresponding reduction of 1kw's worth of coal not being burned. That is not how the grid and power plants work. Eventually, when there is a big enough reduction in demand there could be a corresponding response by not bringing a plant online and for that those types of figures are worth considering. The key is to emphasize reductions in demand as being of primary importance as that is the only thing that will shut down coal fired power plants. These reductions need to be as large as possible as soon as possible. This applies to microgeneration as well. This is why I advocate taking as much of the demand off grid as possible. Simply feeding electricity from microwind or PV into the grid has a similar as using more efficient devices. It is a small reduction demand requiring millions more to do the same to accomplish any reductions in emissions. By going off grid the reduction is far greater, not just because the demand is removed not just reduced, but also because by living off a finite store of electricity one conserves to a far greater degree. Why buy and operate that 50" plasma screen when you can only power it for an hour a week? Reductions are multiplied. Energy is saved by not powering it and by not building it to be transported, sold, used, recycled and/or trashed, in the first place.

Posted by: C Robb on 21 Feb 09

I, for one, would like to see a collection of some of these numbers and figures in order to see how they are connected forward and backward in real life processes.

Any good information for that? Surely some exist...?

Haven't seen any online but in just quotes here and there, disconnected from one another.

Posted by: borki on 21 Feb 09

I'm reminded of two quotes:

"In God we trust, all others bring data" - W. Edwards Deming

"There are three kinds of lies: lies, damned lies, and statistics." - Mark Twain attributed this to Benjamin Disraeli, but Wikipedia gives a somewhat more complex history.

It would be a major service if someone would put up a website that that cataloged such politicized pronouncements but converted them all to the same standard units. For example "Global Warming will cost 80 trillion dollars over the next 100 years" [I just made this number up, prove yours is better!]. But how much is it per American taxpayer per year?

Posted by: George McKee on 21 Feb 09

Post A Comment

Please note that comments will remain open for only 14 days after the article is posted. While previous comments will remain visible, attempts to post new comments after this period will fail. This helps stop comment spam, so your forebearance is appreciated.

The Worldchanging comments are meant to be used for further exploration and evaluation of the ideas covered in our posts. Please note that, while constructive disagreement is fine, insults and abuse are not, and will result in the comment being deleted and a likely ban from commenting. We will also delete at will and without warning comments we believe are designed to disrupt a conversation rather than contribute to it. In short, we'll kill troll posts.

Finally, please note that comments which simply repost copyrighted works or commercial messages will be summarily deleted.

Yes No







MESSAGE (optional):

Search Worldchanging

Worldchanging Newsletter Get good news for a change —
Click here to sign up!


Website Design by Eben Design | Logo Design by Egg Hosting | Hosted by Amazon AWS | Problems with the site? Send email to tech /at/
Architecture for Humanity - all rights reserved except where otherwise indicated.

Find_us_on_facebook_badge.gif twitter-logo.jpg