Solving unconcious bias one step at a time...

November 13, 2016 3:31 PM

Unconscious bias is particularly insidious, and most simple methods developed to address it typically work for only a few days, if at all. Of the approaches tested in a 17,000 person study, only 8 techniques appear to lower bias temporarily, and many of those require the rather troubling method of both raising empathy for the minority group and creating negative associations with the majority group. In fact, simply educating people about negative stereotypes can actually increase stereotyping. The problem is still being worked on, and researchers are optimistic. In the meantime, it can be best to avoid triggering bias at all: blind auditions can reduce bias in settings like orchestras (though maybe not in technical hiring, where the problem appears to be that disadvantaged groups give up sooner). It can also help for people from advantaged backgrounds to speak out when they see racism or bias: a just-published study shows that if a "high-follower white male" calls out a harasser on Twitter, there is a drop in overt racist behavior. You can also become aware of your own biases through the Implicit Association Test.
posted by blahblahblah (32 comments total)

As a white dude, eldest grandchild upon whom weird expectations have been placed, who has called out racist relatives for using certain words, i can tell you that only stopping your relatives from using words is not enough. they may just resent you without changing the way they think.

it s probably more important, in white-only spaces like my thanksgiving table, to lead your white people into empathy. In fact, i stopped worrying the word and have tried to go for the empathy response first, because you only get time to say one thing at the table...
posted by eustatic at 4:19 PM on November 13, 2016 [11 favorites]

Turns out I am explicitly biased against Adobe Flash. Then when I clicked back through a second time it served up a completely different page than the one I had received before, and now I'm wondering if the whole exercise is just screwing with my head. I think the actuality is that my head is done in by current events and am currently registering negative infinity evens.
posted by glonous keming at 4:41 PM on November 13, 2016 [3 favorites]

I probably should have made it clearer in the FPP, but, with the exception of the Twitter study, all of this is about implicit bias - the kind you have without knowing it, and that you will still tend to use in decision-making, even if you try your best to avoid these sorts of biases. Its not the obvious (or even dog whistle) racism/sexism/antisemitism but the kind that creates lots of small biased inferences even when you know better.

Facebook's site on this is surprisingly good at explaining the types of bias.
posted by blahblahblah at 5:02 PM on November 13, 2016 [5 favorites]

This cognitive bias cheat sheet may be helpful
posted by Lanark at 5:08 PM on November 13, 2016 [5 favorites]

I wonder if it makes any difference to the test that you try like hell not to have a bias. I took the Gay vs. Straight test, and my internal self image is pretty strongly tied up in not being biased against LGBTQ folks; I scored no bias, but that seems somewhat suspect to me, 'cause as bad as I want to be unbiased, I recognize the unlikelihood.
posted by Mooski at 5:10 PM on November 13, 2016 [5 favorites]

Mooski - if you answer every answer slowly because you are thinking about it, the test won't show results. Also it is worth pointing out that the IAT has strong critics that say that it shows nothing. There are lots of other ways of measuring implicit bias as well, like audit studies (this one is particularly depressing).

I would also say, contra Lanark, that implicit bias should be considered separately from other forms of cognitive biases (though that list is cool, thanks). Most of those are also ways in which people make bad decisions based on heuristics, of course, but they don't have the same societal effects as implicit racial/gender/etc bias. Considering implicit bias directly is important, I think, in a way that considering the Halo effect is not. Implicit bias (called implicit association) is on that list though.
posted by blahblahblah at 5:24 PM on November 13, 2016 [6 favorites]

This cognitive bias cheat sheet may be helpful

Well, replication efforts in the area of cognitive biases are not going well. At this point I'd probably look for something written by someone a bit more qualified than "I read Wikipedia."
posted by pwnguin at 5:57 PM on November 13, 2016 [4 favorites]

At this point I'd probably look for something written by someone a bit more qualified than "I read Wikipedia."

So, that infographic codex (at the bottom of the link) has little merit? I disagree. Bash Wikipedia all you like, critical readers explore the citations and their own cross-reference. Moreover, you're not an Addams fan? Please name another collaborative resource. What's worthy of dismissive criticism is Jstor and others as the web becomes wholly commercial versus informational.


Anyway, this particular metric of muscle motor response, rapidity, and "switching" is problematic, but I do 'em anyway. They remind me of the twitch games I grew up on.
posted by lazycomputerkids at 7:33 PM on November 13, 2016

I'm an IAT skeptic. I think the stuff it leaves out about racism and sexism is much more important than the stuff it explains. To me, it is part of an anti-political ethic of personal virtue that is not working out for us. I'll leave it at that.

BUT I should say that the advice for "high-follower white males" from the study sounds excellent (and has no particular connection with implicit bias).
posted by grobstein at 7:44 PM on November 13, 2016 [1 favorite]

I wish you'd explain a bit, grobstein - why should it lead to virtue ethics only? It's just measuring the comparative strength of associations (via speed) and assuming some sort of (probably emergent) underlying categorical architecture. The precise depth, structure, and permanence of the associations it claims to measure haven't been worked out, as far as I know, and what actually goes into those categories is still thought to happen at the level of culture, as far as I understand. And I don't think anyone's suggesting that's not amenable to change.

If implicit bias contributes to awfulness, seems like it's useful to try to understand it.
posted by cotton dress sock at 8:22 PM on November 13, 2016 [1 favorite]

Echoing a few points above, I have never understood why implicit bias is something we should be deeply concerned about, or why the inability to reduce implicit bias is a huge problem. Most of us do not have jobs where we make split-second life-or-death decisions (and even police put themselves in these situations far, far more than they ought to). Instead, the main effects of conscious and unconscious racism operate at normal or even excruciatingly slow speed -- resumes rejected, housing denied, policies and candidates voted for, etc. These sorts of things can most certainly be overcome with training and practice, to recognize, confront and amend not lightening-fast biases, but the sorts of systematic harms we do to each other over long, repeating time-scales. If I can learn to change my long-term behavior towards racial justice, what does it really matter if my stupid basal brain still insists on miscategorizing faces given 300 ms to decide? It would seem more essential to focus on changing these long-term behaviors that are under our conscious control and have the most important effects, rather than spending so much time worrying about deep-seated but transitory twitch responses.
posted by chortly at 9:11 PM on November 13, 2016 [6 favorites]

This is a great post.

So I remember when I was in college in the late 90s, I saw a study that some students there had done that had been picked up in a few places. It was that when news programs were reporting on crime, if the suspect was black they would run the story with a photo (usually a scary looking mugshot), way more often than they would if the suspect was white.

Now, we are much more aware of that kind of bias in reporting, but then it was kind of a revelation to me - I had no idea. Even worse, I realized that this particular example was only one tiny piece of the millions of tiny messages we get across a lifetime on things like race and gender.

I also remember that it was probably around that time that Ellen officially came out as gay on her show. Ellen is super likeable and I think for the millions of Americans who had never known one gay person, she became a face for gay people that seemed safe for them to like. And after that, I felt like we started to see a lot more gay characters across mainstream media and that's when we started to see a pretty rapid (in terms of changing minds and attitudes) change in things like acceptance of gay marriage.

So I think the media has a huge role to play here. The Nautilus article linked in the FPP mentions this:

The most successful methods linked African Americans with positive stories or images or whites with negative ones, and they tended to be vivid and highly personal.

I think we often see this with political partisanship as well. When we talk to someone about something that may seem abstract to them (something, for example, like food stamps) they may have very strong negative opinions about people who "cheat the system", but if they're shown stories of people who receive government benefits and the real life circumstances that they face, they soften their stance.

We know that racial minorities are hugely underrepresented in pretty much every way in mainstream media (as actors, actresses, journalists, reporters, commentators, etc). And while we may have a fair number of visible women in media, how many of them fall outside of fairly narrow parameters, physically? How many movies pass the Bechdel Test?

Ino our society, it is very, very easy to fall back on what you see or hear about whatever group we're talking about. And if you live in an area where you rarely or never come in contact with nonwhite people, and the only representations you're seeing on TV are on the news when they commit a crime because black people are continually underrepresented in tv, movies and ads; then you're probably pretty vulnerable to a Donald Trump going on TV and shouting dogwhistley things like "inner city crime" and "Chicago". Because you have no good counterexamples that come to mind to counteract that message. Media representation matters. And not just the people in front of the camera. People of color and women will often report on things like police profiling and sexual assault differently than white men would, and framing also matters.

So I do think the media plays quite a big role in this. A lot of the things we use now to counteract implicit bias don't really work for more than a few days because why would they? When people go from the training room right back into a culture that prioritizes the experiences, voices and lives of white men above all others, and reinforces that in probably hundreds of little ways, every single day? To remember to consider your own unconscious bias in all your interactions would require a ton of near-constant conscious effort, so it's no wonder that it fails.

Of course representation in the media would not magically cure all implicit bias. The LGBTQ community is much more diverse than just Ellen, obviously, and I get that there are many, many experiences that are not represented at all, and they face their own forms of bias. But I do think that breaking down the initial walls that people have against people that are different from them is huge, and the only way we can even begin to do this is by not only broadening the kind of experiences that we're seeing, hearing and reading about, but also by broadening the pool of people who are telling the stories; then we can start to replace the stereotypes we hold with real stories and faces and lives.
posted by triggerfinger at 9:11 PM on November 13, 2016 [10 favorites]

These sorts of things can most certainly be overcome with training and practice, to recognize, confront and amend not lightening-fast biases, but the sorts of systematic harms we do to each other over long, repeating time-scales.

Recognition and confrontation is what this twitch test purports to do. Its aggregation accords a distribution. Most people believe they have a sense of humor and are attentive in bed. Accord a distribution, and... it just isn't true...
posted by lazycomputerkids at 9:24 PM on November 13, 2016 [1 favorite]

Does the IAT randomly select whether it starts by pairing light&good/dark&bad or light&bad/dark&good? It seemed to me to mostly be testing how quickly you can re-learn a different set of keybindings after learning the first set.
posted by straight at 9:36 PM on November 13, 2016 [3 favorites]

...how quickly you can re-learn a different set of keybindings...

Now, that's kinky stuff right there.
posted by lazycomputerkids at 9:39 PM on November 13, 2016 [3 favorites]

Please name another collaborative resource.

Open Science Framework allows scientists to collaborate and distribute data. Replication Project: Psychology for example, contains the replication reports mentioned before. It's not exactly citizen science, but then neither is editing a wiki. I have no reason to think that Wikipedia is particularly good about annotating previously settled science with sufficient asterisks. Especially when so many motivated parties are involved.

And lets not kid ourselves that there's no economic stakes at play. Every discussion about reproducibility brings up tenure cases at risk, and PhD students whose placement is in jeopardy if a result is disputed. And certainly, it'll be more difficult to get NSF funding to do further research on an effect regarded as non-existent by peers.

Does the IAT randomly select whether it starts by pairing light&good/dark&bad or light&bad/dark&good? It seemed to me to mostly be testing how quickly you can re-learn a different set of keybindings after learning the first set.

It does randomize, and the argument is that its harder to learn things that your implicit bias wires the other direction. So which ever pairing you're slower at is the pairing you're biased with.
posted by pwnguin at 10:25 PM on November 13, 2016 [1 favorite]

Instead, the main effects of conscious and unconscious racism operate at normal or even excruciatingly slow speed -- resumes rejected, housing denied, policies and candidates voted for, etc.

True, but I think we should be concerned about the extent to which these slow processes are the product of a series (or even a single), snap decision which is made and then retrospectively justified. A significant part of my job is challenging decisions about housing provision made at a local governmental level. Two things are apparent after perusing hundreds of files of disclosed documents: (1) a lot of important decisions are made on the basis of gut instinct by the very first person to encounter an applicant, and (2) once a decision has been made against an applicant, institutional inertia acts to justify and reinforce that decision (decisions in favour of applicants are more prone to later organisational reversal). One can see how, given the tendency for negative decisions to stick, even very short term biases in information processing may become part of a systemic culture of racism, even without any conscious bias at any time.
posted by howfar at 11:08 PM on November 13, 2016 [11 favorites]

It's not exactly citizen science, but then neither is editing a wiki. I have no reason to think that Wikipedia is particularly good about annotating previously settled science with sufficient asterisks. Especially when so many motivated parties are involved.

Yer HemHawin'

You bashed an aggregation of Wikipedia sources as trifling and when challenged [thank you] reply with a presentational platform of wikis, abstracts, and spreadsheets. You complain that Wikipedia's not Particularly Good? By what metric? Your imagination? Finally, your double-down is a reasoning that "settled science" is populated by un-motivated parties? No meaningful human enterprise you'll name is without motivated parties. And, yes, I understand you mean competing interests.

As an organized reference by terms of range and scale, Wikipedia's editors have developed what Addams imagined and its detractors are in dire need of Arts courses and a spankin' with a taxonomy paddle.
posted by lazycomputerkids at 12:24 AM on November 14, 2016 [1 favorite]

Wikipedia makes no pretence of being authoritative in and of itself - the whole point is that everything in the Wiki is supposed to be referenced to a more trusted published work elsewhere. It’s entirely reasonable to suggest that one of the places that Wikipedia falls down is when previously settled science becomes regarded as less trustworthy within the field - there are still all those lovely authoritative sounding works to refer to & the in-field debunking of the original studies doesn’t (yet) have the kind of authority that Wikipedia likes.

The replication crisis in psychology is a thing & I believe that the implicit bias association studies are amongst the ones that have failed to replicate in independent follow up studies using the same test setup as the original papers.
posted by pharm at 2:01 AM on November 14, 2016 [1 favorite]

I believe that the implicit bias association studies are amongst the ones that have failed to replicate in independent follow up studies using the same test setup as the original papers.

Is that so? Do you have examples of these replication failures?
posted by cotton dress sock at 5:12 AM on November 14, 2016

It does randomize, and the argument is that its harder to learn things that your implicit bias wires the other direction. So which ever pairing you're slower at is the pairing you're biased with.

I find it hard to believe anybody goes faster on the second set of pairings right after getting habituated to the first set.
posted by straight at 8:29 AM on November 14, 2016

In fact, apparently there's some research to that effect.

Researchers must be careful in interpreting IAT effects because of the cognitive inertia inherent to the IAT task. Cognitive inertia denotes the difficulty of switching from one categorization rule in the first block to the opposite categorization rule in the second block. Response latencies are slower in the block that is administered second, and IAT effects depend on the order in which the two blocks are administered. When the faster block comes first, cognitive inertia slows down responses in the subsequent block, thus enlarging the IAT effects. When the slower block comes first, cognitive inertia slows down responses in the faster block and thus decreases the IAT effects.

The authors demonstrate that cognitive inertia can eliminate (Study 1) and even reverse (Studies 2 and 3) IAT effects. Cognitive inertia distorts individual IAT scores and drastically reduces correlations with predictor variables. In Study 4, the authors eliminate cognitive inertia effects by manipulating the block order repeatedly within instead of between subjects.

posted by straight at 8:33 AM on November 14, 2016

Let my rephrase my comment. I made two errors in the Male Female science/humanities set. However the problem of right hand, left hand dominance was the cause of the errors, and the cause of differences in speed. There are some things that imply bias, that speak more of neuro-physical limitations.
posted by Oyéah at 9:48 AM on November 14, 2016

Is that so? Do you have examples of these replication failures?

"Predicting ethnic and racial discrimination: A meta-analysis of IAT criterion studies", Oswald et al appears to be the obvious place to start - there’s a fair amount of back and forth in the literature after that, but the core result that IAT doesn’t have the strength claimed in the original papers appears to be uncontested - the discussion appears to be centred on what effect the remainder has in the real world.
posted by pharm at 11:04 AM on November 14, 2016 [1 favorite]

(this is just from a quick scan of the literature, so a deeper dig might turn up more papers / discussion in both directions)
posted by pharm at 11:05 AM on November 14, 2016

I did find this April Fools article from the British Psychological Society this year that made me chuckle though.
posted by pharm at 11:11 AM on November 14, 2016

True, but I think we should be concerned about the extent to which these slow processes are the product of a series (or even a single), snap decision which is made and then retrospectively justified. A significant part of my job is challenging decisions about housing provision made at a local governmental level. Two things are apparent after perusing hundreds of files of disclosed documents: (1) a lot of important decisions are made on the basis of gut instinct by the very first person to encounter an applicant, and (2) once a decision has been made against an applicant, institutional inertia acts to justify and reinforce that decision

I certainly agree this example is one to be concerned about and another good example where quick decisions have long-term implications. So I certainly didn't mean to say that the IAT stuff has nothing to be concerned about. But again, most of us aren't making hundreds of gut decisions in short periods of time. If I am reviewing resumes, for instance, I have time to examine each, note the gender and ethnicity of the applicant, second-guess my gut, and try to make a better overall decision. It certainly takes more time and attention, but it doesn't require fixing something so deep-seated and elusive as implicit associative racism. My gut can be as racist as it wants as long as my brain can keep it in check. None of that is to say that reducing implicit racism isn't important, since it may always creep out in ways we don't anticipate. So maybe this is just a critique of the media framing (as well as the framing in many of the papers themselves) that suggests that implicit bias is (a) the deepest measure of racism, and (b) that it is really hard to fix. These papers would have a lot less impact if the authors just said, yeah, people have these implicit biases, but they really don't matter too much in most walks of life if you learn to think carefully about race and second-guess your decisions based on basic theories of social justice and fairness. Getting people to do the latter is hard enough, but at least it's not as mysteriously biologically challenging as retraining the brainstem.
posted by chortly at 11:14 AM on November 14, 2016

My gut can be as racist as it wants as long as my brain can keep it in check.

You raise some really good points, chortly, but the important thing I've always taken from the implicit bias research is just the fact that it exists. The only way for your brain to keep it in check is if you know there's something to keep in check -- otherwise, you don't do the necessary second-guessing of yourself, and you just go with your gut.
posted by forza at 3:02 PM on November 14, 2016 [2 favorites]

My gut can be as racist as it wants as long as my brain can keep it in check...but they really don't matter too much in most walks of life if you learn to think carefully about race and second-guess your decisions based on basic theories of social justice and fairness.

Modeling bias by the metaphor of a "gut" isn't compelling to me: As though bias and dissonance were a tennis match in which the players and the ball are all discrete and distinguishable entities readily accounted. Nor would simply "two" places in the brain be a compelling description.

That said, I'm as skeptical as anyone having posted about keybindings and interpreting motor skill "latencies" versus "hesitations" with arbitrarily associated values. I dunno: Taking a measure of a rapid, physiological response to address the unconscious? Is it as much chicanery as a Scientology E-meter?
posted by lazycomputerkids at 3:09 PM on November 14, 2016 [1 favorite]

Mod note: A couple of comments deleted. Please don't pick a fight in here about the PC police.
posted by LobsterMitten (staff) at 3:27 PM on November 14, 2016 [1 favorite]

Can someone point me to a study or studies showing a relationship between IAT results and real-world behavioral outcomes? Thanks in advance.
posted by fraxil at 6:26 PM on November 14, 2016

Researchers must be careful in interpreting IAT effects because of the cognitive inertia inherent to the IAT task. Cognitive inertia denotes the difficulty of switching from one categorization rule in the first block to the opposite categorization rule in the second block. Response latencies are slower in the block that is administered second, and IAT effects depend on the order in which the two blocks are administered. When the faster block comes first, cognitive inertia slows down responses in the subsequent block, thus enlarging the IAT effects. When the slower block comes first, cognitive inertia slows down responses in the faster block and thus decreases the IAT effects.

Yeah - while randomization of a large number of trials may smooth out that effect this makes it seem somewhat misleading to present individual scores from a short series as meaning anything in particular.
posted by atoxyl at 11:09 PM on November 14, 2016