Something's Wrong but You'll Never Know What It Is

June 21, 2010 1:17 PM

In search of the Dunning-Kruger effect: filmmaker and New York Times opinionator Errol Morris talks with Ig Nobel Prize-winning psychology professor David Dunning about, variously, lemon juice-wearing bank robber McArthur Wheeler, the concept of "unknown unknowns" popularized by former US Secretary of Defense Donald Rumsfeld, Scrabble greatness, anosognosic paralysis, and how "our incompetence masks our ability to recognize our incompetence". (first of a five-part series) (previously)
posted by Doktor Zed (51 comments total)

Previously. (You couldn't be expected to know that, but you could have known how to know you didn't know that.)
posted by The Bellman at 1:23 PM on June 21, 2010 [2 favorites]

I knew your lemon juice link was actually downloading a PDF each time I clicked it instead of jumping to a page. Really. I just wanted 10 copies so I could pass them around.
posted by hal9k at 1:29 PM on June 21, 2010 [5 favorites]

former US Secretary of Defense Donald Rumsfeld

Former Poet Laureate more like!
posted by GuyZero at 1:39 PM on June 21, 2010 [2 favorites]

Something's wrong with me because I don't know which link is the interview
posted by sswiller at 1:43 PM on June 21, 2010

found it
posted by sswiller at 1:44 PM on June 21, 2010 [3 favorites]

I've always been baffled by people making fun of Rumsfield for that "unknown unknowns" remark since it made perfect sense to me (making fun of him for everything else he's ever done is fine with me). I stopped trying to defend that comment in conversation a long time ago because no one ever was remotely convinced it was actually an intelligent comment. Glad to see someone else make the same point.
posted by Falconetti at 1:55 PM on June 21, 2010 [29 favorites]

I recognize that I suck at many things, which indicates that I'm actually some sort of renaissance man.
posted by East Manitoba Regional Junior Kabaddi Champion '94 at 1:58 PM on June 21, 2010 [1 favorite]

Yet, when arrested, Wheeler was completely disbelieving. “But I wore the juice,” he said. Apparently, he was under the deeply misguided impression that rubbing one’s face with lemon juice rendered it invisible to video cameras.

Neighbors complain he blames the juice for everything.
posted by hal9k at 1:58 PM on June 21, 2010 [2 favorites]

Something's wrong with me because I don't know which link is the interview

The Dunning-Kruger effect has been mentioned many times on Metafilter, e.g. here, here, here, here, but I don't think it's ever been better illustrated than my own post on the topic. (At least, I don't think it has.) I can hardly wait for the next four parts of Morris's interview so I can further contemplate the scotoma of my own incompetence.
posted by Doktor Zed at 2:00 PM on June 21, 2010 [1 favorite]

Worth reading alone for "epicaricacy". But it's a fantastic piece even without that.
posted by Lemurrhea at 2:08 PM on June 21, 2010

This is why all of the best students that I ever worked with also had the most self-esteem problems.
posted by oddman at 2:09 PM on June 21, 2010

Rumsfeld should have been a philosopher, he was clearly a talented epistemologist who accidentally found himself in the military-industrial complex and failed upwards into being SecDef. (Similarly to Boris Johnson, a man who's stumbled into being Mayor of London and is obviously depriving an Oxford college of its brilliant classics professor).
posted by Electric Dragon at 2:15 PM on June 21, 2010

Falconetti: I've always been baffled by people making fun of Rumsfield for that "unknown unknowns" remark since it made perfect sense to me...

It's not that it didn't make sense. For me, at least, it was funny because it was an incredibly wordy and pedantic way to say something trivially true, obvious, and thoroughly unhelpful, and a complete reversal for an administration that had been insisting for a year that they knew absolutely for certain what was happening in Iraq before we had feet on the ground there. It wasn't shocking because it was wrong, it was shocking because he felt it necessary to explain at length that two adjectives and two nouns could strung together in four ways. It was a horrifying glimpse into what passed for Deep Thought inside the administration.

I think many of us felt it was waaaay to similar to this.
posted by Western Infidels at 2:16 PM on June 21, 2010 [7 favorites]

This is why all of the best students that I ever worked with also had the most self-esteem problems.

Your best students weren't smart enough to realize they were smart? How were they your best students, then?

Simmer down, I know what you meant, just a joke. ;-)
posted by Cool Papa Bell at 2:18 PM on June 21, 2010

WAKE UP SHEEPLE. THE DUNNING-KRUGER EFFECT IS A LIBERAL CONSPIRACY TO STEAL YOUR MEDICARE.
posted by schmod at 2:19 PM on June 21, 2010 [4 favorites]

I've always been baffled by people making fun of Rumsfield for that "unknown unknowns" remark since it made perfect sense to me (making fun of him for everything else he's ever done is fine with me). I stopped trying to defend that comment in conversation a long time ago because no one ever was remotely convinced it was actually an intelligent comment. Glad to see someone else make the same point.

Honestly, I think it's an intelligent observation. The problem is that it was deployed as a way of brushing aside legitimate questions from a member of the armed forces about how much preparation and planning had gone into a nation's occupation.

When the question boils down to, "Why did you not plan for this, when people warned you that you needed to plan for this?" it's kind of weak to reply that there are lots of unknowns.

It was philosophically sound, but used in a situation where it was inappropriate.
posted by verb at 2:23 PM on June 21, 2010 [4 favorites]

Re: Rumsfeld's quote

Go back and read the exchange, because it's worse than most remember.

The quote was a response to a rather simple question: "...is there any evidence to indicate that Iraq has attempted to or is willing to supply terrorists with weapons of mass destruction? Because there are reports that there is no evidence of a direct link between Baghdad and some of these terrorist organizations."

I can't read this as anything other than a yes/no question. You either have evidence or you don't. You can, of course, deflect the question ("our evidence is classified").

But no, rather than do his fucking job and answer the question, Professor Rumsfeld decided to take a half-assed you-don't-know-what-the-fuck-you're-talking-about potshot. In direct response to the question above, he said:

Rumsfeld: Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns -- the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.

And so people who have the omniscience that they can say with high certainty that something has not happened or is not being tried, have capabilities that are -- what was the word you used, Pam, earlier?

Q: Free associate? (laughs)

Rumsfeld: Yeah. They can -- (chuckles) -- they can do things I can't do. (laughter)
posted by Cool Papa Bell at 2:33 PM on June 21, 2010 [4 favorites]

Is that this guy's excuse?
posted by Salvor Hardin at 2:35 PM on June 21, 2010

Mod note: If this is the first time you've noticed me leaving a little administrative comment like "hey, I added the Dunning-Kruger effect interview link to the post, carry on", you may find that you start noticing that sort of thing a lot more in the near future; that's attributable to the Baader-Meinhof effect. That both these bits of neuroscience have come up in the same place is no coincidence, and is attributable to the fact that hyphenated psychological phenomena travel in packs, an observation known in the literature as the Speeshus-Bulkrap effect. Carry on.
posted by cortex (staff) at 2:37 PM on June 21, 2010 [8 favorites]

Thank you for the link clean-up, Cortex. Failing to notice the missing link to the actual interview would have been a great meta-joke about the Dunning-Kruger effect in action in an FPP, but unfortunately I'm not clever enough to have thought of it in the first place.
posted by Doktor Zed at 2:47 PM on June 21, 2010

The Boondocks take on Rumsfeld's speech by the character Gin Rummy voiced by Sam Jackson.
posted by Grimgrin at 2:54 PM on June 21, 2010 [1 favorite]

McArthur Wheeler is such a great story.
posted by Pope Guilty at 3:09 PM on June 21, 2010

this story needs to be preserved for posterity. you need to read past the "but I wore the juice" line.

Wheeler had walked into two Pittsburgh banks and attempted to rob them in broad daylight. What made the case peculiar is that he made no visible attempt at disguise. The surveillance tapes were key to his arrest. There he is with a gun, standing in front of a teller demanding money. Yet, when arrested, Wheeler was completely disbelieving. “But I wore the juice,” he said. Apparently, he was under the deeply misguided impression that rubbing one’s face with lemon juice rendered it invisible to video cameras.

In a follow-up article, Fuoco spoke to several Pittsburgh police detectives who had been involved in Wheeler’s arrest. Commander Ronald Freeman assured Fuoco that Wheeler had not gone into “this thing” blindly but had performed a variety of tests prior to the robbery. Sergeant Wally Long provided additional details — “although Wheeler reported the lemon juice was burning his face and his eyes, and he was having trouble (seeing) and had to squint, he had tested the theory, and it seemed to work.” He had snapped a Polaroid picture of himself and wasn’t anywhere to be found in the image. It was like a version of Where’s Waldo with no Waldo. Long tried to come up with an explanation of why there was no image on the Polaroid. He came up with three possibilities:

(a) the film was bad;

(b) Wheeler hadn’t adjusted the camera correctly; or

(c) Wheeler had pointed the camera away from his face at the critical moment when he snapped the photo.

this is the scientific method in action:

1) Formulate hypothesis: lemon juice renders me invisible to cameras.
2) Design a experiment to test your hypothesis: take a picture of myself with a polaroid after rubbing lemon juice on my face.
3) Run your experiment: Hypothesis confirmed! I took a picture of my face and it was not visible.
.
.
.
4) Profit!

Also see P K. Dick's book: Portrait of a Crap Artist.
posted by ennui.bz at 3:16 PM on June 21, 2010 [4 favorites]

Anyone who has ever had to read unsolicited manuscripts for a living at any time is aware of this effect. Most submissions are more or less normal. A small percentage are good to brilliant. And then there's another small percentage on the other side, which are not only bad, but so bad that it is *baffling* that anyone could have possibly thought that it was worth submitting; clearly, who ever sent it had so little self-awareness that they could not understand what was wrong with what they had done, even though it seemed to be glaringly obvious.

My favorite recent one of those was the one-page lesbian tap-dance fetish piece submitted as a full-length play.
posted by kyrademon at 3:19 PM on June 21, 2010 [7 favorites]

Metafilter: one-page lesbian tap-dance fetish piece submitted as a full-length play.

*ducks*
posted by warbaby at 3:28 PM on June 21, 2010

And then there's another small percentage on the other side, which are not only bad, but so bad that it is *baffling* that anyone could have possibly thought that it was worth submitting...

I know the feeling, but sometimes I think there's more going on behind the scenes.

I work in the videogame industry. One time, at E3 (a major tradeshow, if you've never heard of it), a mother came up to me with her teenage son, and asked if her son could speak to me. I agreed, and boy started going off about his wicked cool game design and asked if my company would agree to consider it.

Now, the answer, for various legal reasons, is always supposed to be not just "no" but "hell no." But for some reason, this kid's pitch read funny to me. I mean, dude had his mother bring him to E3. Something was ... off.

I decided to break the rules. I was overly polite and gave him my business card. The mother thanked me profusely and said her son really, really loved games.

Two weeks later, I get his email and a link to his "Web page."

I put that in quotes because it was barely a site. Think geocities circa 1997, but this was 2003. It was barely cribbed together. And the game design itself was ... well ...

It suddenly hit me that the boy was developmentally disabled. Probably PDD NOS, which is why he could carry a conversation for 5 minutes, but needed his mother's help to get around.

And of course his mother knew that. But she was fulfilling his wish. They had trooped to E3, which is some gamers' version of Mecca. The visuals and stimulation likely blew him away. He had spoken to a developer. He had delivered a pitch. He had sent an email. He got a "thank you" email back from me.

Who cares if his game design was more akin "outsider art" than anything real? I hope I made him happy for one little, teeny-tiny moment.

Certainly made me happy to think I might have done a tiny little good deed for someone, just by being polite.
posted by Cool Papa Bell at 4:32 PM on June 21, 2010 [8 favorites]

I have an interview with the president of the Alcor Life Extension Foundation, a cryonics organization, on the 6 o’clock news in Riverside, California. One of the executives of the company had frozen his mother’s head for future resuscitation. (It’s called a “neuro,” as opposed to a “full-body” freezing.) The prosecutor claimed that they may not have waited for her to die. In answer to a reporter’s question, the president of the Alcor Life Extension Foundation said, “You know, we’re not stupid . . . ” And then corrected himself almost immediately, “We’re not that stupid that we would do something like that.

lol.
posted by rosswald at 4:55 PM on June 21, 2010

I stopped trying to defend that comment in conversation a long time ago because no one ever was remotely convinced it was actually an intelligent comment

It wasn't an intelligent comment, it was the worst kind of obfuscating. He should have been saying "you were right, I was wrong, we fucked up." Instead he continued to pretend no one ever questioned the facts presented as the basis for invading Iraq, and that the only prudent move was to bomb the shit out of Baghdad in case Saddam Hussein and Osama Bin Laden are seekrit BFFs. Cuz, you know, you never know, right?

Rumsfeld demonstrated with that line in the clearest possible terms that it doesn't matter what you or I think. "Unknown unknowns" was so stupid because he could only have used it subjectively (hey, I didn't know) given the mountains of evidence suggesting his take on the situation was kinda bullshit.
posted by Kirk Grim at 5:11 PM on June 21, 2010

> Worth reading alone for "epicaricacy".

Actually, that was the one part that made me groan. As I said here:

[Schadenfreude is] in the latest edition of Merriam-Webster's Collegiate unitalicized, which means it's a naturalized English word by the only useful standard (people's individual feelings about a word not being a useful standard). And "epicaricacy" is not a word by any meaningful definition, unless you count "being used once in an 18th-century dictionary" as a meaningful definition.

Aside from that, it's Morris's typically readable and stimulating job, and I look forward to further installments.
posted by languagehat at 5:15 PM on June 21, 2010 [1 favorite]

One of the more useful models I ever learned hanging out on Usenet writing groups was the idea that your writing skills and your evaluative skills don't develop in tandem with each other. Sometimes your writing skills outstrip your evaluative skills and you think you're awesome. Sometimes your evaluative skills outstrip your writing skills and you think you're awful. But neither of those says anything about where your writing skills actually are.
posted by Jeanne at 5:19 PM on June 21, 2010 [3 favorites]

I find it a little funny that this concept was "discovered" and named after Dunning and Kruger. In The Republic, I believe Plato attributed the following quote to Socrates, "As for me, all I know is that I know nothing." I remember learning this in high school Latin class, and I thought it was a bit of a revelation. Along those same lines you often hear people say things like, "a little bit of knowledge is a dangerous thing." The bank robber in the interview knew enough about the scientific method to conduct an experiment, but not enough to conduct a meaningful experiment. I think humanity has been well aware of this sort of cognitive bias for millenia if not longer. The real problem is that there are individuals who are just too stupid or simply incapable of understanding such a simple concept. Perhaps the inability to recognize this cognitive bias is part of the pathology of stupid.
posted by kscottz at 5:28 PM on June 21, 2010

“But I wore the juice,”

But but but but, what?
posted by nola at 5:44 PM on June 21, 2010

I like the juice.
posted by Splunge at 6:15 PM on June 21, 2010

It wasn't an intelligent comment

My experience was more that people treated it as a nonsensical comment. Maybe "intelligible" is a more neutral term to use. I agree that it was a bullshit answer to the question but it wasn't gibberish as a lot of people pretended it was (I'm talking about in casual conversation with friends, I don't remember what the pundits said about it).
posted by Falconetti at 6:17 PM on June 21, 2010

Socrates, "As for me, all I know is that I know nothing."

Montaigne, "The only thing certain is nothing is certain."

Karl Popper, "We know nothing—that is the first point. Therefore we should be very modest—that is the second. That we should not claim to know when we do not know—that is the third."

Arcesilaus, "that he knew nothing, not even his own ignorance,"

Pyrrhonism (founded by Sextus Empiricus) was based entirely on this line of skepticism.

But, uh, what else are academics going to do if not name things after themselves and proclaim themselves the experts? (I say this only half kidding, but you must see a little irony in them not knowing that not knowing existed before them? Errol Morris is a sharp guy, I have no doubt he'll pick up on this.)
posted by geoff. at 6:24 PM on June 21, 2010 [3 favorites]

but you must see a little irony in them not knowing that not knowing existed before them?

There's a bit of a difference between conceiving of the idea that unselfaware ignorance is a force in the world and doing formal research on the subject. I haven't gone through all the links but to the best of my knowledge Dunning hasn't declared that he discovered the mere idea that people sometimes don't account for their own lack of knowledge.

"The Dunning-Kruger effect" has, at least, the advantage of being snappier and more specific than "that thing that Socrates said about being aware of one's own ignorance" or "that Rumsfeld thing but seriously he kind of had a point".
posted by cortex at 6:30 PM on June 21, 2010 [2 favorites]

The one thing I've learned in grad school is knowing how well I know what I know (and all the associated corollaries). This is precisely what makes it difficult for me to carry on conversations with my neighbors, coworkers and assorted co-speciests. Of course, it could just be undiagnosed PDD-NOS, which would explain a lot about my son.
posted by mollweide at 7:22 PM on June 21, 2010

It wasn't an intelligent comment, it was the worst kind of obfuscating.

It was actually quite revealing, albeit unintentionally. What it revealed was kind of disturbing: that one of the most powerful men in the world was influenced in his thought processes by a cultic group, Werner Erhard's Landmark Forum. Unfortunately I can't find a good link to back me up but the wording is supposedly straight out of Erhard's teachings. It's unfortunate that there are people out there following his hokum, but someone with as much power & influence as Rumsfeld? That idea still gives me the willies.
posted by scalefree at 7:42 PM on June 21, 2010

Last time I read about the D-K I filed it as "the dumbest ones are the most certain"... which was reassuring in a way.

This reading reveals more troubling detail. So now it's "if one is dumb, they'd be too dumb to know."

The strikes me as both obvious and *really* unhelpful.

Am I dumb? "Yes!" ah ha! I knew I wasn't. See? But wait... that means. Oh god. No, see that's proof right there. Yr safe! Phew. Hold on a sec... !?!!

/reaches for TV remote
posted by MeatLightning at 8:22 PM on June 21, 2010

This is why it is a good idea to move away from people who are doing something that doesn't make sense and who say "I know what I'm doing" when asked about it.

This was impressed on me very forcefully by a fellow who told me of working in a hard rock mine in Idaho and watching a guy tamp a dynamite cartridge with an iron crowbar. His last words were "I know what I'm doing." Whereupon my informant quit his job and moved away from Idaho.

Ever since I heard that story, when someone says "I know what I'm doing," I make sure to increase my distance from the speaker.
posted by warbaby at 8:43 PM on June 21, 2010 [1 favorite]

Explains George Bush perfectly:

“Unskilled and Unaware of It: How Difficulties of Recognizing One’s Own Incompetence Lead to Inflated Self-assessments,”
posted by batboy at 8:43 PM on June 21, 2010

1. Morris already made the Socrates connection in the footnotes.

2. The second part is out http://opinionator.blogs.nytimes.com/2010/06/21/the-anosognosics-dilemma-somethings-wrong-but-youll-never-know-what-it-is-part-2/
posted by mad bomber what bombs at midnight at 9:14 PM on June 21, 2010

This is getting complex. Just why I like Morris.
posted by warbaby at 9:58 PM on June 21, 2010

Why do yuo dumfucks like Errol Morris my YouTube channel is better than his shit
posted by benzenedream at 10:20 PM on June 21, 2010

When I was building houses, the crew I was on had, at least, noticed the pattern of incompetence. If/when someone was doing something incorrectly, you went over to them and said "No, not like that, you do it like this." This, of course, sounds reasonable. The only problem is, whenever someone would utter that terrible phrase, they would injure themselves; doing whatever they were doing.

Whether it was from frustration, or rushing the action, the injury was immediate and related. Not a jinx, as such, but we got to saying "No, not like that, you do it like- ah! Hah hah! Not this time! See that? Almost got myself. Get $OtherWorker to show you how to do that."

Injuries dropped dramatically. At least amongst the tenured staff.
posted by LD Feral at 7:02 AM on June 22, 2010 [2 favorites]

On a related note, here's a Slate interview with former financial speculator and hedge fund manager (and catastrophic two-time loser on Wall Street) Victor Niederhoffer on the topic of being wrong. "In retrospect," he admits, "I know much too much about errors and much too little about the opposite, whatever it is."

After discussing the Thai stock market crash of 1997, George Soros's random predictions, and the importance of a good mentor, he moves on to a bête noir of his:

Let's turn [mentoring] on its head for a second with one of my favorite topics, the hoodoo. There are certain people you meet in life who are like the locomotives that always used to blow up—people who, wherever they go, disaster always ensues. [...] And they generally have a string of failures behind them, they generally are in need of capital, they generally talk a much better game than they play. And they often flatter you and pretend to be your very amiable friend before they really know you. Hoodoos are very good at what they do. A lot of times they command the center of attention and they try to dazzle you with the trappings of success—which when you look into it you find is a will o' the wisp.

These bringers of bad luck that Niederhoffer tries to avoid are not con artists as such, however. They sound instead like walking examples of the Dunning-Kruger effect, diligently promoting themselves beyond their abilities, yet apparently never reflecting on either their limitations or their failures.
posted by Doktor Zed at 10:54 AM on June 22, 2010

Is there a term for the extreme version of this, where a person is terrible at something but not only is not aware of this, they think they are awesome at it? I've had at least one boss like that.
posted by Legomancer at 1:22 PM on June 22, 2010

Legomancer: hubris and self-delusion are probably more common than organic medical conditions.

The fact that your instances are in positions of power supports hubris as the reason.

Sort of like this millionaire I know who is a complete disaster at business, but his family keeps bailing him out with more money. He thinks he's hot stuff and deserving of admiration. I apply the same behavior to him as I do to people who say "I know what I'm doing" - immediate and total avoidance.

It's interesting that so many people in comments (here and at the Times) suggested GW Bush as a presidential example of the social condition, but Morris was talking about the medical condition and Wilson.

The really tough part is determining if the subjective reality matches the objective one.

That's always the tough part. Fool or dupe? Con man or cheat? Myth or hoax? Psychopath or true believer?
posted by warbaby at 8:58 AM on June 23, 2010

In Part 3, Morris asks whether Woodrow Wilson was an anosognosic case after his second-term stroke - possibly further enabled by his wife and doctors - and if so, did that ultimately play a part in keeping the United States out of the League of Nations? (In characteristically wry fashion, Morris also introduces the hagiographic multiple Oscar-winning wartime "Wilson" (1944) into his exploration of how we view events and draw conclusions.)
posted by Doktor Zed at 10:33 AM on June 23, 2010

In Part 4, Morris interviews neurologist V.S. Ramachandran, author of/lecturer on "Phantoms in the Brain", on whether an "anosognosic patient knows (on some level) about the paralysis", which Ramachandran addresses via the idea of layered belief.
posted by Doktor Zed at 11:25 AM on June 24, 2010

And for the fifth and final part in the series, Morris goes back to Dunning to ask some follow-up questions (such as what a Venn diagram of cluelessness, self-deception and denial would look like) and learns the secret origin of the lemon juice-wearing bank robber from a former Pittsburgh detective who worked on the case.
posted by Doktor Zed at 10:38 AM on June 25, 2010