The Debunking Handbook

January 3, 2012 2:39 PM

"Although there is a great deal of psychological research on misinformation, there's no summary of the literature that offers practical guidelines on the most effective ways of reducing the influence of myths. The Debunking Handbook boils the research down into a short, simple summary, intended as a guide for communicators in all areas (not just climate) who encounter misinformation." Direct PDF link.
posted by brundlefly (33 comments total)

It's a hugely interesting area, the recent apparent retreat from rationality. Why is complete nonsense perfectly ok in the political sphere these days? A very important question.

These two articles published just today further illustrate the syndrome:
Age of the Amateur
Krugman - nobody understands debt
posted by wilful at 3:15 PM on January 3, 2012 [3 favorites]

PDF, bah, don't believe anything that's not in PowerPoint.
posted by sammyo at 3:31 PM on January 3, 2012

Why is complete nonsense perfectly ok in the political sphere these days?

Oversimplyfing, but: I blame the media (Jon Stewart excepted, of course). The handbook is a nice idea, but I'd like to see something about how to recognize BS for yourself, not defuse it in others. In fact, I'd like to see that skill taught as a semester-long course in high school.

Just really simple stuff, like noticing when someone is throwing a red herring at you: "Al Gore claims the climate is warming, but did you know he lives in a really big house and often flies on planes?" If people had the presence of mind to think "maybe Al Gore is some kind of hypocrite...I don't know or much care...shouldn't we get back to this climate issue instead?" -- and if they expected the same of the talking heads on TV, we might be a little better off.
posted by uosuaq at 3:33 PM on January 3, 2012 [4 favorites]

uosuaq: "The handbook is a nice idea, but I'd like to see something about how to recognize BS for yourself, not defuse it in others. In fact, I'd like to see that skill taught as a semester-long course in high school. "

I couldn't agree more. I've often thought that some sort of critical thinking class should be required.
posted by brundlefly at 3:41 PM on January 3, 2012 [1 favorite]

the recent apparent retreat from rationality

Recent?
posted by yoink at 3:42 PM on January 3, 2012 [3 favorites]

Bullshit-detecting would be a hugely fun class to teach in high school. 14 year olds would eat it up. Unfortunately the teacher would probably get a lot of flak from parents over it!

Greatly enjoyed the pamphlet. It ties into the theory-building going on at the Max Planck Institute for Human Development on the cognitive processes people use to make judgments and inferences.

Interesting that it seems the human brain uses speed of memory retrieval as an indicator of trustworthiness. They think it's almost like 4chan's Sleep Sort, where the brain tries to remember everything at once, and whatever gets "returned" first is picked.

Presumably, in natural (non-propagandized) environments, the frequency that you notice something is well-correlated with how often it happens, or what your social group thinks of it. However, in our media culture, gut feelings are being manipulated further and further from being effective.

Systematic propaganda is probably the biggest discovery of the 20th century that will define how the challenges of the 21st century are resolved.
posted by anthill at 3:55 PM on January 3, 2012 [3 favorites]

Recent?

An Outline of Intellectual Rubbish written by Bertrand Russell .....in 1943.
posted by storybored at 4:03 PM on January 3, 2012 [6 favorites]

It's a hugely interesting area, the recent apparent retreat from rationality. Why is complete nonsense perfectly ok in the political sphere these days? A very important question.

In general, ideological resistance to rationality is typically a psychological response, if there is something to protect. If disagreement is presented in a way that is interpreted as antagonistic by one side, the other side will instinctively retreat, regardless of the reason. It's usually not fundamentally an issue of needing to be more rational when one side is being interpreted as speaking condescendingly to the other, and this is often how public discourse is presented, especially within the media. And sometimes one side actually does get pushy because it gets frustrated with the lack of movement from the other on issues that they deem to be very important, which tends to not be effective, but too irresistible to not attempt. Antagonism and truth don't go well together psychologically and it tends to create a cognitive dissonance in people as they try to resolve it, which usually results in writing off a dissenting party than engaging it, regardless of the truth claim.

Additionally, it's pretty typical of either side of an ideological debate to throw statements of irrationality at the other when it comes to differing opinions about issues that are meaningful, especially if one side interprets issues as being force-fed from a position of power. It goes against human nature to be humble and accept it if the other side is prone to gloating (or there is the fear that they will). The response, "well it's good for you regardless of how it's presented" is generally not going to fly, and people will instinctively knock the spoon full of carrots out of the hand that comes in too hard and insistently.

What's a good answer to this problem? I have no idea, except genuine friendship and care for others on the other side of an issue. Genuine ideological change flourishes in environments in which people feel as if they were cared for, not as if they were bullied into something. Unfortunately, this tends to go more slowly than most people have the patience for, and hence, two sides of an issue that never seem to resolve anything, as they want to hurry up and move those "irrational people" along due to the importance of a particular issue. This may seem simplistic, but I do believe it's the mindset that makes people eventually go, well, the other side is incorrigible and unable to see reason, and therefore we must use power to bend them to our will. And when you have this environment where it's more of a power struggle than a discussion, you more often have people writing off rational discussion entirely because they insist that it simply does not work. And the pushing and the prodding and the force-feeding creates an infinite loop of non-rational engagement that pretty much never ends.
posted by SpacemanStix at 4:22 PM on January 3, 2012 [8 favorites]

uosuaq: "The handbook is a nice idea, but I'd like to see something about how to recognize BS for yourself"

Isn't that what an education in the liberal arts was supposed to be for?
posted by klanawa at 4:29 PM on January 3, 2012 [3 favorites]

The handbook is a nice idea, but I'd like to see something about how to recognize BS for yourself, not defuse it in others. In fact, I'd like to see that skill taught as a semester-long course in high school.

You mean like a class to teach the difference between truth and falsehood? Would a whole semester really be necessary?
posted by Edgewise at 4:33 PM on January 3, 2012

'43? get in the way-back machine:

The Natural Science of Stupidity

Stupid runs deep.
posted by Lesser Shrew at 4:45 PM on January 3, 2012

Edgewise: "You mean like a class to teach the difference between truth and falsehood? Would a whole semester really be necessary?"

Certainly. It's not about saying "This is true. This is incorrect." It's about giving people the tools to properly make those determinations themselves. Just one semester may not be enough, but it's a start.
posted by brundlefly at 5:06 PM on January 3, 2012

@klanawa, @Edgewise -- cute, but: (a) klanawa, I'm talking about starting in high school, at the latest, and (b) Edgewise, I'm not proposing some nonsense like an infallible method to discover the truth; I was suggesting we try to give people (kids) a set of skills that would help them recognize when they're being mislead or distracted...ideally, help them tell a good or at least an honest argument from the kind they're likely to encounter much of the time.

Looks like brundlefly got there a little before me, but I'm posting anyway.
posted by uosuaq at 5:09 PM on January 3, 2012

Isn't a "bullshit detection" course basically the same as a rhetoric class? I understand that rhetoric used to be a fairly common class, but it was not by the time I made it to school.
posted by feloniousmonk at 5:13 PM on January 3, 2012

The equivalent to rhetoric that I did have in high school was a year-long "debate class" which was basically a way to allow the debate team to work during school hours but also included the basics of policy debate as well as Lincoln-Douglas (basically, empirical vs. values, if you're unfamiliar). It was probably the only useful class I took in high school.
posted by feloniousmonk at 5:15 PM on January 3, 2012

I think you need more than just critical thinking/rhetoric instruction. There's also teaching people about cognitive biases.
posted by storybored at 5:16 PM on January 3, 2012

Interesting, I didn't know that pareidolia was considered a cognitive bias but it certainly makes sense. If people were aware of it, I think SyFy and History Channel would go bankrupt.
posted by feloniousmonk at 5:19 PM on January 3, 2012

I can't speak to the content of a high school rhetoric class, never having seen one. To me, personally, "rhetoric" is about enhancing your argument with powerful language. It's essentially neutral -- you could use rhetoric to lead an army to war for the wrong reason, or the right reason, or no reason at all; you could use it, as MLK did, to keep people's spirits up during the civil rights movement; or as the NRA does, to keep donations flowing in against a nonexistent threat.
What I'm talking about is something else, though.
posted by uosuaq at 5:26 PM on January 3, 2012

Anti-propaganda education was only needed in the 1950s, when them Communists were using it.
posted by anthill at 5:28 PM on January 3, 2012

I can certainly see how a class in rhetoric would aid in identifying rhetorical techniques used by others, but I don't think that by itself would be sufficient. It would certainly be one good tool among others.
posted by brundlefly at 5:33 PM on January 3, 2012

A big of being able to make an effective argument is understanding how the components of one fit together, and if you understand those, you can identify when someone is doing something subversive with their own argument, which is why I think rhetoric would be valuable.

I think that it's as important, if not more important, to teach how to make good arguments as it is to teach how to detect bad ones.
posted by feloniousmonk at 5:34 PM on January 3, 2012

But sure, I'm not trying to argue it's the silver bullet, rather just that this stuff used to be part of the traditional education but no longer is.
posted by feloniousmonk at 5:35 PM on January 3, 2012 [1 favorite]

You mean like a class to teach the difference between truth and falsehood? Would a whole semester really be necessary?

If anything, a semester would be too short. You could devote your entire life to studying rhetoric, philosophy, logic, etc.

What's strange is that the only course I had in grade school which was anything like a cognitive bias class was DARE, which is the pervasive American anti-drug program that otherwise does nothing at all useful. There was genuinely decent material mixed in there about recognizing fallacies in alcohol and cigarette advertising.

An actual course in Critical Thinking would be pretty great. I'd love if it could be a year-long course, with the first semester being geared towards rhetorical exercises and the second semester being more about long, sustained pieces of well-reasoned argument.
posted by Sticherbeast at 5:37 PM on January 3, 2012

What's strange is that the only course I had in grade school which was anything like a cognitive bias class was DARE, which is the pervasive American anti-drug program

Interesting you bring that up. I consider myself to have been a smart kid who knew how to think for himself. I had hippie parents and grew up taking the kind of classes where having your own opinion was OK and even expected.

And yet, DARE is the one thing in my youth where I can look back and say: "Oh my God! I was brainwashed!" I really was. I really believed pot was addictive and horrible and was genuinely scared to be near it. And I believed drug users deserved to go to jail. When you look at the prison situation in our country rationally, DARE is evil. Period. It's as close to a Hitler Youth program as we have in America*, and I'm not sorry for the comparison.

*Well, was, up until "See something, say something."
posted by drjimmy11 at 6:11 PM on January 3, 2012 [2 favorites]

Edgewise, I'm not proposing some nonsense like an infallible method to discover the truth; I was suggesting we try to give people (kids) a set of skills that would help them recognize when they're being mislead or distracted...ideally, help them tell a good or at least an honest argument from the kind they're likely to encounter much of the time.

I know, I just couldn't resist. Even so, it's a tall order, and hard to get into without taking a position on real issues.
posted by Edgewise at 6:15 PM on January 3, 2012

I read the pamphlet with great interest and think that there's some seriously useful information in it. Two thoughts:

1) You can use the techniques in this pamphlet to spread falsehoods if you're wanted.

2) Every time I think about our human tendency to believe false things, I think of Animal Farm and hope that I'm the old, cynical Benjamin the donkey who realizes how the farm is being betrayed by the pigs and not one of the sheep that hasn't noticed that Squealer has replaced the seven commandments have been replaced with one.
posted by Joey Michaels at 6:28 PM on January 3, 2012

I see the whole aggressive ignorance in three lights. First, it reminds me nothing more than the newspaper clips from the 19th century. There always have been political scandals, and these seem more of the same. What's the difference? No difference.

Second, people always thought things were getting worse, that we had it better off back in the day, that the world was going down hill. Have a little faith you guys, we're always going to think that. That's part of politics.

Third, I see a growth in this type of thinking as kind of necessary if we really want to stamp it out. It's the same as how consumerism has become so bad that I take every opportunity to buy for life from local craftsmen. People will get so fed up with this blather that they will tune it out in favor of online news that tells things clearly. Or they won't - see point two.
posted by rebent at 7:40 PM on January 3, 2012

You mean like a class to teach the difference between truth and falsehood? Would a whole semester really be necessary?

I was thinking at least two semesters. Bullshit can have a lot of dimension. Or, as SpacemanStix pointed out, while Level 1 may be about how to recognize the bullshit that's getting dumped on you, Level 2 would be how to reconcile the bullshitter without just perpetuating the divisive Us-vs-Them-ism which is the toxic ground where bullshit breeds.

What's a good answer to this problem? I have no idea, except genuine friendship and care for others on the other side of an issue. Genuine ideological change flourishes in environments in which people feel as if they were cared for, not as if they were bullied into something. Unfortunately, this tends to go more slowly than most people have the patience for, and hence, two sides of an issue that never seem to resolve anything

Nicely put, sir.
posted by philip-random at 8:55 PM on January 3, 2012

How about the basic laws of stupidity as a starting point?
posted by sneebler at 9:54 PM on January 3, 2012 [2 favorites]

philip-random: Bullshit can have a lot of dimension. Or, as SpacemanStix pointed out, while Level 1 may be about how to recognize the bullshit that's getting dumped on you, Level 2 would be how to reconcile the bullshitter without just perpetuating the divisive Us-vs-Them-ism which is the toxic ground where bullshit breeds.

I'd really recommend Asking the Right Questions: A Guide to Critical Thinking, I've read quite a lot of books on critical thinking/fallacies/biases (mostly to try and stop myself making irrational mistakes, because I do tend to), and whilst I learned a lot of rules and their Latin names, I wouldn't say that I learned to think critically per-se.

ATRQ does teach that (and a methodology/process to follow), I was getting completely stumped by some of the exercises the first time through. As it turns out there are sometimes quite subtle errors to be made (and half errors, and almost truths), it isn't always just a 6 foot straw man in the middle of the room pointing out that Al Gore is fat.

It also covers the whole bit about being non-confrontational (us v. them) with it, trying to get everyone comfortably to the truth, not just being a dick and using your knowledge of the rules to cudgel your opponent into submission and prove that you're the big (wo)man.

[/effusion]
posted by titus-g at 10:09 PM on January 3, 2012 [1 favorite]

I think you can learn a lot if you print out these Cognitive Biases Flash Cards and flip through them while arguing about anything.
posted by twoleftfeet at 10:54 PM on January 3, 2012 [1 favorite]

PDF of Harry Frankfurt's 'On Bullshit'

Frankfurt's essay gave me the courage to use the word "bullshit" in many of my academic papers. So far, much to my dissatisfaction, no professor has ever called me out on it.

Wikipedia overview for those not wanting to go to the PDF just yet.
posted by Knigel at 11:03 PM on January 3, 2012

Edgewise et al.:

Being a IB student I can't resist: there is such a course, for high school students, part of the International Baccalaureate Programme. Theory of Knowledge It's generally taught in two or three semesters, IIRC. Best course I've ever taken, and that includes college.
posted by thegears at 3:21 PM on January 4, 2012