Cancel
Advanced Search
KEYWORDS
CATEGORY
AUTHOR
MONTH

Please click here to take a brief survey

The Fallacy of Examples, and the Problems of Extrapolating from Media
Ethan Zuckerman, 6 Jul 08

David Weinberger has an intriguing post up today about the “Fallacy of Examples“. He’s reacting to a column from Nick Kristof in the New York Times titled “The Luckiest Girl“, which recounts the story of Beatrice Biira, a young woman from Uganda whose improbable journey through Connecticut College began with the donation of a goat to her family through Heifer International.

David finds the story moving - how could you not! - but points out that Biira’s amazing journey is hardly a typical outcome of livestock donation programs. Indeed, the reason Kristof is telling it is that it’s so remarkable. And that may be something of a problem:

I’ve noticed in business writing in particular the frequency of what we can call the Fallacy of Examples (a type of Fallacy of Hasty Generalization). You read some story about a successful CEO as if we should learn from his (yes, usually it’s a him) example. But we are struck by examples frequently because they’re exceptional. As exceptions, examples are the last thing you want to learn from.

Not always, though. Sometimes examples are typical. That’s different. The trick is determining which are which.

The problem of deciding whether an example is typical or exceptional struck me as resonant with Clay Shirky’s new (brilliant, must-read, go buy it now) book Here Comes Everybody. Throughout the book, Clay points out that online communities tend to experience a power-law (Pareto) distribution of participation. If you attempt to generalize about the group as a whole from the most prolific participants, you’re going to misunderstand what’s going on.

This is a predictable misunderstanding - we appear to have a tendency to assume that people we encounter are distributed on a bell curve. Fly into Amsterdam and you’ll notice that there are a lot of tall people around. Spend a day or two and you’ll likely conclude that Dutch people are tall, significantly taller than Americans. This turns out to be true - Dutch people are now roughly two inches taller than their American counterparts, likely due to a better diet and excellent state-subsidized healthcare - your extrapolation from a few data points is a pretty accurate one.

Try a different experiment - watch some American TV and try to extrapolate the bell curve of body type in the US. You’re going to get it wrong, and you’re going to feel fat, no matter how skinny you happen to be. People on American television aren’t a bell curve distribution in term of weight - they’re way, way out on an extreme. Media critics suggest that the relentless repetition of images of underweight actresses has a negative impact on young women, leading them to aspire to extreme body types.

Here’s the thing - it’s lots easier to write about extreme examples rather than median ones. (It’s probably easier to watch extremely thin people on TV than ones of median weight as well.) Stories of prolific wikipedians, alpha bloggers or brilliant flickr photographers are more interesting than stories about someone who set up a LiveJournal, posted five times then gave up… which is lots more typical. And Biira’s story is far more compelling than the story of a girl whose family got a goat, and is slightly better fed than the median Ugandan, but who didn’t get to go to school. This, unfortunately, is probably closer to the median effect of livestock donation - not a bad thing, by any means, but not wholly transformative.

The answer to the Fallacy of Examples is not to stop giving examples. Human beings need stories to be interested in issues - that appears to be how we’re wired to take in information. Joi Ito, writing about the recent Global Voices summit, talks about how personal stories can help solve “the caring problem”, making international incidents relevant to audiences who might not care about this news otherwise. Kristof needs to tell us about Biira to get us interested in livestock donation - we’re not going to pay attention without a human story to hold onto.

(Indeed, some critics point out that livestock donation is a form of storytelling as well. Your $120 isn’t buying a goat - it’s a way of getting you to donate to an agricultural charity which will use your money to provision livestock, but also to pay staff salaries, fundraising expenses, etc. The story of giving a goat to a poor family convinces you to give, and perhaps to give more than you otherwise would.)

The solution may be to try to contextualize the story - is the example given an ordinary or an extraordinary one? Kristof signals this with his title, making it clear that Biira is an extraordinary case. But the story would probably be a fairer one with a more representative, median example, offered as a contrast. If you buy a goat for a Ugandan child, you’re probably not going to send a young woman to college… but you just might. It’s hard for me to blame Kristof for telling this amazing story, but it makes me wonder how many unconscious and inaccurate generalizations I’m making every day, looking at extremes and unconsciously assuming they’re medians.

This post originally appeared on Ethan's excellent personal blog My Heart's in Accra

Bookmark and Share


Comments

This is thought-provoking. It's related to irrational financial decision making - selling when stocks are losing money (when actually that's the time to buy), buying lottery tickets, et al. I read the Kristof op-ed piece and having often donated to Heifer (it was the coolest thing around before Kiva!) I liked knowing that some folks in their programs to hit the jackpot. But the odds of even a well-prepared low income American student getting a full-ride at Connecticut College are pretty low!


Posted by: Betsy - GreenMicrofinance on 6 Jul 08

There is exceptional good things that happen and also exceptional bad things. Both get used as examples and though I think the exceptional good often leads people off course, the exceptional bad things like 9-11 are what seems to really send people way outside of their normal rational way of thinking. Everyone would like to avoid traumatic events, but traumatic events do not make for good examples anymore that very fortunate events do. I don't mean to reduce the meaning of significant events, but a hundred year storm is not something to plan for every day.


Posted by: Jarrett on 6 Jul 08

This article caught my eye because I posted a link to "The Luckiest Girl" on my blog as soon as it was released. A great point is made, but in many ways I think it's a moot point. It's certainly not the worst thing in the world if this heart-warming article inspired readers to donate and Kristof does make note that this is an extraordinary circumstance.
I have to agree that stories add a particular power to any philanthropic effort, but keep in mind as well the importance of giving something that has meaning. It's not just that giving a goat is a story but that you chose it. When I was young. my mom never let me give cash as a birthday present because it's a hollow gesture. It was a valuable lesson on the real meaning of generosity and I don't see why we should treat people like Beatrice any differently. Add to this the fact that giving a goat ensures the family benefits from our donation and not corrupt politicians or warlords.


Posted by: Brendan on 7 Jul 08

Hah,
as a sort of clay shirky fanboy,
i read a few sentences of this article, and I was just about to recommend parts of Here Comes Everybody as a conversation about that problem exactly.

I'm happy to see at a closer look that the article was more or less about the book!

But I think the power-law distribution that the book discusses is not about good or bad examples, it's about the structure of community participation in collective action, it's about how the negligible cost of the failure of a huge percentage of contributors is what enables the successes of the few, it's about how the community performs as an interrelated eco-system that can collectively cover a huge landscape of improbable possibilities that expensive participation could not previously cover.

In a sense, I think that a Power law distribution has nothing to do with the assumed goal of an organization like Heifer International. There are lucky people and lucky girls everywhere, in any situation, in any country. Lucky people are not special to the Ugandan recipients of goats, and they don't make a case for the success of a program, at all.

Heifer International is presumably not playing venture capitalist, scanning the landscape of Ugandan people for the 3 or 4 that are worth following through and supporting. It shouldn't care about a power law curve, if there happened to be one. It is presumably trying to raise the overall bar for every single person that it reaches. A story of a relevant example would be the MEDIAN case, and the rest of the metrics are comparatively irrelevant.


Posted by: Kunal Gupta on 7 Jul 08



EMAIL THIS ENTRY TO:

YOUR EMAIL ADDRESS:


MESSAGE (optional):


Search Worldchanging

Worldchanging Newsletter Get good news for a change —
Click here to sign up!


Worldchanging2.0


Website Design by Eben Design | Logo Design by Egg Hosting | Hosted by Amazon AWS | Problems with the site? Send email to tech /at/ worldchanging.com
©2012
Architecture for Humanity - all rights reserved except where otherwise indicated.

Find_us_on_facebook_badge.gif twitter-logo.jpg