Cancel
Advanced Search
KEYWORDS
CATEGORY
AUTHOR
MONTH

Please click here to take a brief survey

Smarter Cities and Flawed Rankings
Alex Steffen, 16 Jul 09
Article Photo

Smarter Cities is an NRDC project designed to support urban transitions toward sustainability. It's a good project:

"When thinking about the urban environment, more often than not problems come first to mind. Less commonly thought about is the potential presented by cities, potential to rethink and reshape their environments responsibly. Today urban leaders — mayors, businesses and community organizations — are in the environmental vanguard, making upgrades to transportation infrastructure, zoning, building codes, and waste management programs as well as improving access to open space, green jobs, affordable efficient housing and more. If they succeed in making their cities more efficient, responsible and sustainable, what will result will be smarter places for business and healthier places to live."

The way the project has chosen to go about encouraging the potential of cities to "rethink and reshape" is to publish rankings of cities and their green efforts, based on a set of criteria which on the surface would seem to be great signs of progress -- things like green buildings, green space and recycling rates. But there's a problem.

I've explained before why I'm skeptical of city rankings to begin with: what's measured by these rankings tends not to be a good set of indicators of whether these cities as a whole are actually improving in any meaningful way. And Smarter Cities in particular seems to have gotten the wires crossed between its excellent mission and its flawed measurements.

Seattle, for instance, comes in at #1 in the rankings. Living in Seattle, I feel no qualms about probing into how a city with profound sustainability problems managed to make it to the top of a national ranking for "smart cities." I can tell you it ain't pretty.

Though sustainability itself is a somewhat slippery concept, there are absolutely standards by which we can judge progress, as they mean the same things everywhere, and are pretty good measurements of overall impact. What, for instance, are a city's per capita greenhouse gas emissions? How many miles a day do its citizens drive? How large is their average home and how compact are their communities? How much water do they use? How much energy? How much solid waste do they generate? These sorts of numbers actually tell us something about how the people live, and about their overall levels of impact.

But Smarter Cities counts more easily-measured, but sort of pointless data. For instance, the green building ranking rated the number of Energy Star and LEED buildings in a city, rather than quality of the general building code: so a city like Seattle, where building codes are far behind those of the U.K and Northern Europe, still comes off looking good because it has a few more individual green buildings than other cities.

Similarly, "energy production and conservation" was rated by solely by the percentage of green power sources for its electricity, not total direct energy usage (much less total embedded energy usage). This means that a city like Seattle -- with a highly auto-dependent population, which wastes more or less about as much energy as other Americans (more than the average Californian, and far more than the average German or Japanese) -- looks great, because of the region's abundance of hydropower, while in fact not being particularly ahead of the curve in any other way. We happen to have rain and mountains, so we're "green," never mind the landfills full of dead appliances and the smog hanging in the sky.

Or take transportation, which the rankings defined by the percentage of people who use public transportation, and the number of transportation choices available to the average citizen. Better would have been to compare vehicle miles traveled per capita, which actually measures what's most important, which is how many trips people take in their cars (a number which drops rapidly with density). The bizarre ranking criteria produced the effect of having Los Angeles come in as greener than New York City, despite the fact that New Yorkers drive far less and emit fewer transportation-related greenhouse gasses.

For "standard of living," they employ in part the National Association of Home Builders Housing Opportunity Index, a flat measurement of the cost of housing compared to average wages -- a figure often use to argue for sprawl -- rather than incorporating better understandings of the true cost of living in given communities, such as the Housing and Transportation Affordability Index. Cheap homes don't necessarily mean affordable lives.

"Recycling" was weighed, giving points for the number of things that could be recycled, and for the percentage of waste recycled, completely missing the essential points that recycling a higher percentage of a larger waste stream can still leave a family sending more and more to the dump, or that recycling itself is not a perfect or cost-free solution. A much more interesting measurement would have been solid waste generated per person; a more cutting-edge study would have attempted to get at the real root of the problem, unsustainable consumption itself.

The point here is not to pick on Smarter Cities (or Seattle). The point here is that unless we start defining real success (and measuring our progress in light of it), comparative measurements are worse than useless: they can even become a form of greenwashing. Many, for instance, argue that Seattle's environmental performance (when you take away the hydro and the mild climate) is actually sub-par, but the accolades of others make it hard to hold elected officials feet to the fire over this city's lack of density, low standards and continuing auto-dependence.

I look forward to a city ranking that does the opposite: that makes it easier for individuals to measure their own efforts, easier for citizens to judge progress, and easier for cities to set goals that might in fact make them truly bright green place to live. A truly smarter city would judge itself not by its neighbors, but by what's needed to save the planet.

Bookmark and Share


Comments

I look forward to Worldchanging's city ranking that does all of these things. ;)


Posted by: Jeff on 17 Jul 09

I agree. We don't need yet another half-baked rating system that is based on the most easily available data for every city (lowest common denominator). Also, we have to stop grading on a curve - Seattle, SF or Portland may be doing the best, but are they doing enough? We should not be making comparisons to each other, we should be using a benchmark. We need a grading system based on what it will take to cut our GHG emissions by 80% or get to 350ppm or whatever standard you want to use - but it is not going to be a pretty picture when every city gets Ds and Fs.


Posted by: TDAPDX on 17 Jul 09

Great points, Alex. I'd skimmed over the NRDC rankings recently and had many of the same criticisms. I was particularly irked that they used "Standard of Living" to mean what would actually better be termed Affordability when one of the biggest realizations in city planning this last decade has been the recognition that a high quality of life (cultural offerings, sense of place, public spaces, etc.) isn't just a feel-good urban amenity -- it's integral to making more sustainable communities, i.e., communities where people will enjoy living in medium-to-high densities, walking, bicycling, engaging with neighbors, etc.

Indicators are often flawed, especially so when it comes to sustainability (there was a sizeable literature on this even seven years ago) -- see last year's Brookings report that laughably listed Honolulu with the lowest per-capita carbon footprint. But they're important tools, and we need to make them better. So I'll join with Jeff and say Yes, bring on the Worldchanging Rankings! -- you and your team are plenty qualified to do it.

Daniel Lerch
Post Carbon Institute


Posted by: Daniel Lerch on 17 Jul 09

Good to post this. I am always skeptical.
In Canada, Vancouver was rated #1.
It is so congested with traffic, homelessness and crime are very high and there aren't even washrooms at the skytrain stations or bus loop centres.


Posted by: myna lee johnstone on 17 Jul 09

As a Seattleite, I am very disappointed in the NRDC on this. According to the EPA, Seattle is rated as one of the worst places for air quality in the US. The American Lung Association gave our county an F last year for air quality.

It is ridiculous how far off the NRDC rating system must be - we even had a reporter here publish a story saying that Seattle should breathe easier because of NRDC's ranking. NRDC should clarify this point because our current mayor is touting it as a major success, even though he is leading the charge to build a mega-tunnel for cars at the expense of public transit options.


Posted by: Becky Stanley on 17 Jul 09

As a Seattleite, I am very disappointed in the NRDC on this. According to the EPA, Seattle is rated as one of the worst places for air quality in the US. The American Lung Association gave our county an F last year for air quality.

It is ridiculous how far off the NRDC rating system must be - we even had a reporter here publish a story saying that Seattle should breathe easier because of NRDC's ranking. NRDC should clarify this point because our current mayor is touting it as a major success, even though he is leading the charge to build a mega-tunnel for cars at the expense of public transit options.


Posted by: Becky Stanley on 17 Jul 09

Apparently LA's good rating vis-a-vis NY is because its Mediterranean climate means there is less need for heating and aircon. Reasonable, then. Either local climate needs to be factored out, or people need to stop living at high latitudes.


Posted by: Rollo on 18 Jul 09

Austin, San Diego, Dallas, etc. were ranked as the top cities for public transit?--they were ranked "high" in that category.

In 2007 in Austin less than 5 percent of people used public transit to commute. Twenty two large US cities had a greater ridership percentage. About 3 percent in Austin in 2007 walked and less than 1 percent used bikes to commute. Thirty large US cities had greater combined walking and biking.

I question the weighting and scoring--what exactly does "high", "medium" and "low" mean?

As someone who has devsied and written a book on US city sustainability rankings (How Green is Your City? The SustainLane US City Rankings), I know they are not inexpensive or easy to do, but we should expect more at this point in time, especially from such a well-funded operation like NRDC.


Posted by: Warren on 18 Jul 09

Hi Warren, and others,

A superb book on this topic is "Sustainability and Cities" by Newman and Kenworthy. I was just reading it yesterday, so it was fresh in my mind when I read this Worldchanging post. The book is solidly data-based, but also easy to read. It also compares and contrasts cities from around the world, avoiding a US-only view. I highly recommend it.


Posted by: Geoffrey Mantel on 19 Jul 09

I think this could be one of those rare chances you could put your readers in a position to contribute to World Changing. Challenge them to build the ranking as on online tool and post it here next week, your readers from various cities across the country can see how they stack up. I agree, the year over year numbers for the same city are measures of the real progress. We can no longer sugarcoat our lack of progress with a few meaningless token green initiatives--its proving to be greenwash in this town.


Posted by: Sean Howell on 19 Jul 09

I can see your point. I´ve been compiling and writing on city rankings lately and I can even a previous issue which is that these exercises keep a backup of theory on how they conceptualize whatever they rate. That´s why the image they offer is always subjective. For instance, when we see rankings such as livable cities or best cities to work, most of the times the cities ranked first are those with highest prices for housing. So, the most unaffordable cities are being over-rated for a situation that should make them be in lower positions.


Posted by: Manu Fernandez on 20 Jul 09

Thank you for your comments and criticism and for the opportunity presented here to discuss ways to improve Smarter Cities’ research. As background, this year's Smarter Cities report was conducted independently of the NRDC and only recently found hosting at the NRDC website. In preparation for future research rounds, we are developing an open source approach to encourage public discussion of criteria, data sources and methodology. A Smarter Cities wiki for this discussion will be in place by the beginning of August. We encourage you to participate in what promises to be a valuable process.

For our current scoring and criteria, please see “How We Scored Cities” and “Data Sources and Point System” at http://smartercities.nrdc.org/rankings/scoring-criteria. We appreciate all the suggestions for criteria that we have received and are finding measurements of per capita power consumption, CO2 output, waste output and other measurements. The Smarter Cities wiki will serve to gather comments and suggestions on ways to improve the criteria and study methodology for the next research round. All responses are welcome.

Although much of our data was derived from the EPA, the Census, the DOE and other comprehensive sources, survey responses did have a significant result on scores in several criteria, as noted in the Data Sources section. While our survey response rate was a fairly high 24.3%, we want to push for much higher response rates in the years to come.

Last year was the first year we considered cities with populations below 100,000, which added over 400 cities to our list but presented a problem in locating comprehensive data sources that included them. Regarding transportation, we looked specifically at the American Public Transportation Association ridership figures, but because they covered only a portion of the cities we considered, we chose not to weigh them so heavily that the majority of cities lacking them would be unfairly penalized. Instead, we gave more weight to a survey response asking about the types of transportation options available (including bicycle paths, bicycle sharing programs, bus systems, carpool lanes, car sharing, dedicated bicycle lanes, light rail, sidewalks/trails, subways and trolleys). As we prepare for this year’s report, we are revising the data sources and weighting in consultation with NRDC experts and are specifically considering VMT/capita, modal share and walkability.

For the 2009 Smarter Cities report, the Transportation score was composed of two elements: APTA ridership figures, with a maximum of 2 points; and transportation options, with a maximum of 6 points. Los Angeles’ documented ridership for Public Transit (Light Rail, Heavy Rail, Commuter Rail, Bus, Trolley) gave it a score of 2 (for having over 75,989,601 trips in the first quarter of 2007), whereas New York received the maximum 2 points. However, Los Angeles possession of one more transportation option (light rail) than New York gave the city .65 points more. So Los Angeles total score was .15 points higher than New York's. We would agree that in our attempt to be fair to smaller cities the weighting of survey responses was not the best compromise and, as noted above, are shifting to a more pertinent measures for future research.

The “standard of living” score was primarily focused on the affordablity of cities, since without livable wages and affordable housing their environmental benefits may be priced out of reach for many. In previous years, we had asked about living wage legislation, but found this to be too vague a measure. In 2009, the most heavily weighted element was The National Association of Home Builders Housing Opportunity Index, which tallies the total amount of homes sold in a city that would have been affordable to a family earning the median income in that area. We took this as a rough measure of affordability with the addition of Census data on the median household income, the percent of owner-occupied housing and the percentage of families living below the poverty line. For our next report, we are considering adding the Center for Neighborhood Technology’s Housing and Transportation Affordability Index, which includes the affordability of rent, in place of the Census data.

In general, we don’t intend this ranking to be an end in itself but a work-in-progress to identify and spotlight the dozens of cities that are implementing programs to make their towns more efficient, cleaner, more just and more livable, and to get citizens involved in greening their cities and towns. We understand that the research and ranking needs to be improved, but our ultimate aim is to encourage all cities and to engage citizens everywhere in actions that will bring about positive change. In that effort, we invite your participation and would like to hear your thoughts, criticisms and suggestions when Smarter Cities Wiki is launched live in early August or at smartercities@nrdc.org.
Paul McRandle
Consulting Senior Editor
Smarter Cities
NRDC


Posted by: Paul McRandle on 20 Jul 09

It doesn't really matter whether Seattle is ranked "more sustainable" than Milwaukee. The bottom line is that our cities are not sustainable. Why not develop goals (looking forward) versus indices (looking complacently at cities ranked lower than us?). Which city contributes the lowest amount to landfills? Which city has banned plastic bags? Which city has policies encouraging reusing grey water? Which city has tax breaks for "green" construction? Which city's citizens use THE LEAST?

NRDC! Listen up! By these adolescent comparisons - we lose track of the real problem.

the-5th-horseman.blogspot.com


Posted by: the-5th-horseman on 21 Jul 09

The pressing need for a national sustainability framework for U.S. communities is well recognized. Defining and measuring sustainability is a challenge, as you point out Alex, and full of apples to oranges comparisons.

My two cents to this excellent thread is to mention the STAR Community Index, a national, consensus-based framework for gauging the sustainability and livability of U.S. communities. STAR will be launched by 2010, and is currently being developed through a partnership between ICLEI-Local Governments for Sustainability USA, the U.S. Green Building Council, and the Center for American Progress.

STAR will be a rating system, not a ranking system--it will be to cities and counties what LEED is to buildings.

STAR will address the issues you've brought up, Alex, with standardized indicators and metrics that will help local governments set priorities and maximize their investments in strategic actions. They will have the opportunity to “certify” their achievements through independent, third-party verification.

Right now STAR's Technical Advisory Committees are hard at work establishing these indicators and metrics across eight subcategories. STAR's various committees are comprised of leading local government, state, federal, and academic experts (e.g., Sadhu Johnston, Chief Environmental Officer, City of Chicago).

You can learn a lot more about STAR at the ICLEI USA website:
http://www.icleiusa.org/star

Within this page is also a great report from 2008, "A Comparative Analysis of Sustainable Community Frameworks," which is relevant to this conversation.


Posted by: Don Knapp, ICLEI USA on 23 Jul 09

Following up on the comments here, I wanted to mention that Smarter Cities has launched a wiki to discuss, define and strengthen urban sustainability criteria as well as post about individual cities. You can view the wiki and sign up to post and edit at http://wiki.smartercities.nrdc.org.


Posted by: Paul McRandle on 5 Aug 09

THANK YOU! Living in one of the cities which was ranked, I had a keen interest in the outcome. It seems some of our neighbors who own their own coal-fired power plants were ranking higher than us because they are now working to get people switched to "green" energy sources so they don't have to build more fossil-fueled engines. Hardly an accomplishment when they can tap into almost limitless resources to get people to switch.

The NRDC could have done a much more thorough job exploring and weighing information rather than taking it on "wow" value.

Oh well, back to work.


Posted by: Kevin on 24 Nov 09

THANK YOU! Living in one of the cities which was ranked, I had a keen interest in the outcome. It seems some of our neighbors who own their own coal-fired power plants were ranking higher than us because they are now working to get people switched to "green" energy sources so they don't have to build more fossil-fueled engines. Hardly an accomplishment when they can tap into almost limitless resources to get people to switch.

The NRDC could have done a much more thorough job exploring and weighing information rather than taking it on "wow" value.

Oh well, back to work.


Posted by: Kevin on 24 Nov 09

I completely agree with the poster who suggests that while the rankings provide insight, they do little to suggest whether these cities efforts are enough to combat the multitide of issues surrounding sustainability and urbanization. I woudl be interested to compare leading Amercian cities (Seattle, Portland, SF) with world leading cities in sustainability such as Vancouver BC and many European cities. Lets not get caught up in pounding our chests and saying we're number one when som much more could be done in the areas of environmental and social sustainability. I wonder whether social sustainability even factors into these rankings and whether cities are making efforts to develop metrics capable of measuring their own social sustainability.


Posted by: Jim Edson on 21 Jul 10

Post A Comment

Please note that comments will remain open for only 14 days after the article is posted. While previous comments will remain visible, attempts to post new comments after this period will fail. This helps stop comment spam, so your forebearance is appreciated.

The Worldchanging comments are meant to be used for further exploration and evaluation of the ideas covered in our posts. Please note that, while constructive disagreement is fine, insults and abuse are not, and will result in the comment being deleted and a likely ban from commenting. We will also delete at will and without warning comments we believe are designed to disrupt a conversation rather than contribute to it. In short, we'll kill troll posts.

Finally, please note that comments which simply repost copyrighted works or commercial messages will be summarily deleted.

REMEMBER PERSONAL INFO?
Yes No

NAME


EMAIL ADDRESS


URL


COMMENTS



EMAIL THIS ENTRY TO:

YOUR EMAIL ADDRESS:


MESSAGE (optional):


Search Worldchanging

Worldchanging Newsletter Get good news for a change —
Click here to sign up!


Worldchanging2.0


Website Design by Eben Design | Logo Design by Egg Hosting | Hosted by Amazon AWS | Problems with the site? Send email to tech /at/ worldchanging.com
©2012
Architecture for Humanity - all rights reserved except where otherwise indicated.

Find_us_on_facebook_badge.gif twitter-logo.jpg