Some were paralyzed into silence by the request for display.
April 22, 2022 11:27 AM   Subscribe

The Stanford Prison Experiment wasn't actually much of an experiment. The marshmallow study has been justifiably criticized. Perhaps less well known is this informal Labov 1966 paper [PDF] which points out that kids have a lot more to say to a pet rabbit when nobody's watching than to a clinical professional. People behave differently when they know they're being tested. posted by eotvos (38 comments total) 52 users marked this as a favorite
 
Reading through the beginning of that Labov paper, it covers quite a bit more than just pet rabbits! That's actually a fairly fascinating paper and Labov does not hesitate to call out some bullshit when he sees it.

The depressing part to me is how many of the lessons he calls out as being problematic 60 years ago are still happening today.
posted by KirTakat at 11:50 AM on April 22, 2022 [10 favorites]


We've known for years that experimental results in psychology very often don't stand up. And it's not just fun little behavioural experiments, but also studies that have very real impacts on people's health and lives (like the many attempts to re-classify physical illnesses as psychological over the years). Psychology is rotten; maybe not all of it, but enough that we can't trust results in the field.

Is anyone going to do anything or are we just going to keep on pretending that it's a real field of study and letting psychology make a mess for someone else to clean up?
posted by ssg at 12:02 PM on April 22, 2022 [7 favorites]


TIL Shockley was like, super racist: https://en.wikipedia.org/wiki/William_Shockley#Views_on_race_and_eugenics
posted by pwnguin at 12:17 PM on April 22, 2022


Shocking.


Or people codeswitch based on the perceived/expressed cultural disposition or limitation of another person, as they can't always keep the same tone, and perhaps shouldn't. ..but that's probably not what the MM experiment is all about.
posted by firstdaffodils at 12:19 PM on April 22, 2022 [2 favorites]


Thanks for the introduction to William Labov! The linked paper is brilliant in so many ways, and now I want to read all his writing. That will take a while, since apparently he is 94 years old and still publishing.
posted by TreeRooster at 12:44 PM on April 22, 2022 [12 favorites]


Is anyone going to do anything or are we just going to keep on pretending that it's a real field of study and letting psychology make a mess for someone else to clean up?

Aren't lots of people doing manythings all the time, continually making progress over time with further research and information and data because it's a real field of study actively being studied and worked?
posted by GoblinHoney at 12:53 PM on April 22, 2022 [25 favorites]


That Labov paper is distilled and potent:

"I want to give you some indication of how serious these reactions to Headstart's problems are, because you don't have to have a great many psychologists going around saying that it is time for the gas ovens or mass sterilization. All you need is a few, because those who want to apply eugenic foresight only need one or two experts to tell them that what they are doing is scientifically correct and morally justified."
posted by the Real Dan at 1:10 PM on April 22, 2022 [23 favorites]


Aren't lots of people doing manythings all the time, continually making progress over time with further research and information and data because it's a real field of study actively being studied and worked?

Are these problems actually improving? I don't think there is convincing evidence that things are actually getting better. You can't just assume activity necessarily leads to progress. There are plenty of counter examples to that in the history of human knowledge.

There are real harms from bad research. Bad psychological research has made my life and the lives of those I care about significantly worse. This isn't just an academic question.
posted by ssg at 1:51 PM on April 22, 2022 [2 favorites]


Unfortunately, 'further research and information and data' do not cumulatively amount to 'a real field of study'.

Real fields of study depend on institutions having adequate systems of review, and reviewers having a decent amount of expertise and integrity. Such as: peer reviewed journals. (Physics has a major problem with peer review integrity at the moment.) Internal review boards. Reviewers for grantmaking organizations. Annual review journals. Etc. Not to mention grad schools, thesis advisors, textbooks etc.

Real fields of study depend on lots of inside baseball, especially when it comes to methodology. Here's a NYT story about psychology's P-value crisis, for example. And then there's that dead salmon MRI ...
posted by feral_goldfish at 2:00 PM on April 22, 2022 [7 favorites]


Apart from the rabbit study- this phenomenon was one of the major findings of Labov's dissertation, which was on the New York City dialect. He interviewed people and found that as people's attention to their speech shifted- from very careful speech, like reading a list of words, to very casual speech, like recounting a story that they're very emotionally involved in- their speech changed. Specifically, they were more likely to "drop their rs" in casual speech compared to the careful speech.

Labov coined the term "Observer's Paradox" here: casual, vernacular speech is hard to study because the second you tell people you're studying their speech, they start to produce more formal speech that adheres more to standards of "correctness".

If you look around sociolinguistics Twitter, you'll see references to Labov wandering around department stores. This was an exploratory study for the dissertation where he went to a bunch of different department stores in New York (this was in the 60s) and asked various sales people where things were, trying to get them to say "fourth floor". He'd then pretend to not hear them, say "come again?" to get them to repeat. What he found was that the second time around, people were more likely to say their rs compared to the first time, showing that same shifting depending on attention to speech he found in his interviews.

This all might seem obvious now but this was about laying out that vernacular speech has rules and systems- people are systematic about when they use more or less standard speech. And this is really important when you're looking at varieties that are stigmatized and thought to have no rules and are just "bad English" like Hawaiian Creole English or AAE.
posted by damayanti at 2:11 PM on April 22, 2022 [31 favorites]


Oh wow! I had no idea that Labov had also looked at childhood language development. I'm mainly familiar with his work on mapping out N. American dialects and their changes. See this vintage, extremely Web 1.0 webpage, for a summary of some of that work for example.
posted by mhum at 4:07 PM on April 22, 2022 [2 favorites]


And then there's that dead salmon MRI ...

That dead salmon MRI is an example of unleashed popular media spin; the actual experiment is an amusing but reasonable non-standard equipment test, but sometimes the media liked to play up the SCIENTISTS ARE IDIOTS HAHA false angle of the story.
posted by ovvl at 6:12 PM on April 22, 2022 [6 favorites]


Is anyone going to do anything or are we just going to keep on pretending that it's a real field of study and letting psychology make a mess for someone else to clean up?
If we declared things "not a real field of study" because they'd built generations of life-destroying bullshit on a pile of racist assumptions confirmed by poorly-constructed experiments, we'd have to kick pretty much every other discipline to the curb, too. Confronting the problems is a lot harder than just writing off the field of study as a whole.
posted by verb at 6:13 PM on April 22, 2022 [16 favorites]


Oh wow shoutout to my uncle Bill Labov!
posted by Meatbomb at 7:17 PM on April 22, 2022 [25 favorites]


That dead salmon MRI is an example of unleashed popular media spin; the actual experiment is an amusing but reasonable non-standard equipment test

It's a lot more than that. The dead salmon MRI did indeed start out as a test of equipment, more or less. (Instructions were also read to the salmon ...) But the reason the results were publicized by the scientists themselves was because of a counterintuitive outcome, with methodological implications. I.e., if you don't do proper multiple comparisons correction, your results will be as suspect as this dead salmon's apparent ability to read human emotional cues.

, but sometimes the media liked to play up the SCIENTISTS ARE IDIOTS HAHA false angle of the story.

After the blog article I linked to was first published, the lead investigator saw it and presumably thought it respectful enough: he contributed a photograph, and revealed the salmon's ultimate fate. The SCIENTISTS ARE IDIOTS HAHA angle is ... kind of the point? Since the investigators publicized the dead salmon MRI data as a methodological intervention to stop fellow scientists from being idiots.

The authors note that at the time the poster was presented, between 25-40% of studies on fMRI being published were NOT using the corrected comparisons. But by the time this group won the Ignobel last week, that number had dropped to 10%. And who knows, it might, in part, be due to a dead fish.
posted by feral_goldfish at 8:24 PM on April 22, 2022 [7 favorites]


Oh wow shoutout to my uncle Bill Labov!

Since photos of MeFite cats are not obligatory, MeFite uncle anecdotes are likewise optional. But they would be even more welcome than cat pics. I'm just saying.
posted by feral_goldfish at 8:39 PM on April 22, 2022 [6 favorites]


Ooo, I'd also like a link about the Stanley Milgram experiment while we're at it.

I'd also like to point out the excellent podcast You're Wrong About and their episode about the Stanford Prison Experiment.
posted by AlSweigart at 9:50 PM on April 22, 2022 [2 favorites]


The replication crisis is well-known in psychology; science is set up so that young new scientists can make their name by demolishing the work of the titans of their field. What has not yet been solved is demolishing the incentives that have led to coincidental correlations being treated as deep truths about the brain.

One not mentioned here is the Dunning-Kruger effect: this is the famous, deeply satisfying theory that the ignorant are unable to judge the extent of their own ignorance. The problem is that you can also make it fall out of random data: plotting the difference between an expected guess and actual performance, and actual performance, _creates_ a correlation, because "actual performance" is on both axes.
posted by Merus at 10:36 PM on April 22, 2022 [4 favorites]


Since photos of MeFite cats are not obligatory, MeFite uncle anecdotes are likewise optional. But they would be even more welcome than cat pics. I'm just saying.

OK I should just say I only really spent time with him a couple times, but he seemed a lovely man. Some time around / during his wedding to my aunt he gave us a slide show of his Philadelphia neighbourhood / anthropology studies, people sitting out on front porches, dialect / accent observations... they live in a really cool converted carriage house spitting distance to Rittenhouse Square. Maybe 30m long but thin, and the main floor totally open and with huge wooden beams. The rich people's houses on the opposite side got split into apartments, the carriage houses are now the rich people palaces. Kitchen table talk was about the North America dialect map / study he was working on.

Before that my aunt was married to Erving Goffman, I only met him once, but pretty strange because I see both Goffman and Labov mentioned all the time, pretty significant academics both of them, my aunt knows how to pic 'em (and is a distinguished linguist herself - I was amazed as a little boy hearing about her trips into the interior of New Guinea back in the 60s). I remember picking Goffman's own copy of The Presentation of Self in Everyday Life off her bookshelf and asked if I could take it to read it, she offered to buy me a copy... in retrospect kind of a stupid request, obviously an important and valuable relic! Oh, ond my cousin (Goffman's daughter) is Alice Goffman who wrote On the Run.
posted by Meatbomb at 1:04 AM on April 23, 2022 [20 favorites]


Only tangentially related, but since we're sharing personal Labov anecdotes: In a way, Labov actually brought me to metafilter! I had to write a paper for a linguistics class in college, using his method to analyze narratives, and for this paper I used metafilter as a corpus. This is how I got into the habit of reading metafilter, and I'm still at it, almost two decades later. So I have a very positive association to that name. Also, I always liked hearing about his ideas in college. Count me as a fan.
posted by sohalt at 2:41 AM on April 23, 2022 [15 favorites]


Is anyone going to do anything or are we just going to keep on pretending that it's a real field of study and letting psychology make a mess for someone else to clean up?

I take "field of study" to typically refer to the subject matter, not the approach (the geography of the Flat Earth is not a real field of study, no matter how rigorous the approach). I think "human psychology" is a real, albeit ill-defined, thing. I also think that our understanding of it is incredibly limited in comparison to our understanding of most fields subject to some form of systematic scientific study. Given the immense complexity of the systems being investigated, and the fact that we simply don't have explanations for things that seem like they're probably very important (e.g. consciousness), our lack of knowledge makes experimental design exceptionally difficult. The replication crisis has exposed the fact that psychology was doing even worse at this design than we supposed*. But for all that, I think that systematic empirical understanding of human psychology is a valid project, and one in which well-constructed studies can yield valuable results.

But, while I don't think it's the field of study itself that's the problem, I do think that the are some profound and fundamental challenges in psychological research which psychology as a discipline has not acknowledged or accounted for to the necessary extent. In particular, there has been a specific failure to adequately interrogate underlying cultural assumptions about the nature of the mind and self, and a general failure to account for the effect of cultural context on behaviour and thought. The pressure to generate claims that are both interesting and demonstrable creates a perverse incentive to understate the complexity of a field and overstate our understanding. Research has, as a result, often been fundamentally misconceived to the extent of being actively misleading.

These are huge problems, and I appreciate the anger people feel about psychology as a result of these failures. Many of us, particularly those of us who are marginalised, have been failed by systems or people whose behaviour and worldview is significantly shaped by flawed psychological research. As a non-neurotypical person, I am fairly directly subject to the consequences of a number of failures of research design. But I don't think that these failures happened because psychologists are distinctively slapdash, stupid or unethical, but because (as Merus says) of systemic incentives to generate particular kinds of results and offer certain interpretations. Psychology is highly vulnerable to these pressures. I don't think that makes it impossible to address the problems that the replication crisis has exposed, but agree that we can't tackle those problems without dealing with their systemic causes.

* exactly which other fields have similar problems that are less well-known is a matter of ongoing study and debate. I'm personally not at all confident that similar systemic problems are absent in biomed research, and quite possibly other, less obvious, fields. I also think that the apparent slowdown in advancement of knowledge through fundamental research in all fields is probably linked to the same sort of socio-economic incentives that have caused the replication crisis, although I think the symptoms are largely different. My view is that capitalism has become largely inimical to research capable of enabling the kind of fundamentally new understanding that gave rise to the Enlightenment and Industrial Revolution, and which we need to deal with the complexity of the world as we can now observe it.
posted by howfar at 3:42 AM on April 23, 2022 [10 favorites]


You can read Retraction Watch to get a feel of the state of things in all fields.
posted by bleary at 5:42 AM on April 23, 2022 [3 favorites]


Bad psychological research has made my life and the lives of those I care about significantly worse. This isn't just an academic question.

I work for a psychologist whose research has actively made things better for people with chronic health conditions and disabilities. This is also the real world impact of psychology.
posted by jb at 5:58 AM on April 23, 2022 [14 favorites]


The Labov paper is interesting, but I found it weird that the conclusion (to find out what children are thinking and what they know, talk to them directly and simply) doesn't seem to be based on anything presented elsewhere in the paper (children, like everyone, will talk most to people lower in status with them; to get them to talk the most, leave the room completely).
posted by clawsoon at 6:11 AM on April 23, 2022 [1 favorite]


It's worth mentioning that this Labov paper wasn't *meant* to have rigorous methods or conclusions -- it was a talk about his beliefs and principles, backed up with some experiences from the classroom, including this exploration with the rabbit that never turned into a serious publishable-in-a-journal study. Think of it as the academic equivalent of a long op-ed.

(The original reason it came up on Linguistics Twitter was that someone was looking for a copy, and, *since it was just an informal talk,* it was hard to find one, so lots of people had to pitch in and help look.)

He also did a lot of more rigorous work, for which he is justly famous in linguistics. But this isn't it.
posted by nebulawindphone at 7:07 AM on April 23, 2022 [3 favorites]


The challenge for psychology is that is existence depends upon some arbitrary and increasingly hard to defend disciplinary boundaries.

Even assuming it ever made sense before, how can we view (research) psychology today as anything other than a behaviorally-oriented subdiscipline of neurology?

In an era of every-improving psychiatric pharmacology which clinical psychologists don't access, does clinical psychology have a place beside psychiatry as podiatry does to orthopedic surgery (treating a subset of non-/less-interventional disorders) or as RNP/PAs do to MDs/DOs (providing preliminary assessment and routine/low risk treatment). Or does it have neither place?
posted by MattD at 7:14 AM on April 23, 2022


"which clinical psychologists don't access"

Sorry. That's a straw-man argument. Perhaps with a sideorder of no-true-Irishman.
posted by Goofyy at 7:52 AM on April 23, 2022 [2 favorites]


The challenge for psychology is that is existence depends upon some arbitrary and increasingly hard to defend disciplinary boundaries.

Even assuming it ever made sense before, how can we view (research) psychology today as anything other than a behaviorally-oriented subdiscipline of neurology?


Boundary concern isn't unique to psychology.

In an era of every-improving psychiatric pharmacology which clinical psychologists don't access, does clinical psychology have a place beside psychiatry as podiatry does to orthopedic surgery (treating a subset of non-/less-interventional disorders) or as RNP/PAs do to MDs/DOs (providing preliminary assessment and routine/low risk treatment). Or does it have neither place?

Neither - Interprofessional care is a better model for understanding how best to help people with mental illness.

Also, there are some important constraints to psychopharmacology. First, medications must be nontoxic, absorbable and able to cross the blood-brain barrier. Next, the brain uses many receptors in multiple locations, resulting in unavoidable side effects in targeting those receptors. This limits some potential medication options before testing even starts. It also limits some medication options for some people (MAOIs are one example).

Finally, from a practical perspective some people respond better to therapy than medication. It's really helpful to have both available and not from any "subset", "routine/lower risk" perspective.
posted by Emmy Noether at 8:10 AM on April 23, 2022 [1 favorite]


The idea that other fields of study are somehow very valid while psychology is always wrong is silly. I can't even list the amount of mythology, evidence-less tradition and harmful norms that guide medicine. If you want to do work that exists in the real world and not exclusively in the realm of abstraction, you will have extremely messy work. Not to excuse the failures or harm of psychology - they are real - but these failures exist across all sciences and across basically all human endeavors. It seems silly to attempt to throw out the entire practice of attempting to understand and describe human behavior??! We can and and should identify the strands of wrongness and change them.
posted by latkes at 8:14 AM on April 23, 2022 [14 favorites]


brains =/= minds
posted by Meatbomb at 9:40 AM on April 23, 2022 [1 favorite]


What he found was that the second time around, people were more likely to say their rs compared to the first time, showing that same shifting depending on attention to speech he found in his interviews.

I feel like pretending you didn't understand someone is a more specific situation than making someone aware you're paying attention to their speech, & I'm not sure how meaningfully you can generalize their reaction, especially when it comes to enunciating syllables & pronouncing consonants

like I'm all "gum ee lunshlaya yunkum?" in my daily life & if someone indicates they didn't understand me they get treated to a full "i a-M Go-iNG To ea-T LuN-CH La-TeRrr, Do you WaN-T To Co-Me?" in a vaguely BBC accent like I'm Downton fucking Abbey

especially I could see this as a common schema for New Yorkers to have picked up; if the person you're talking to doesn't understand "fourth floor" without the R, very possibly they are not from New York, try pronouncing the R & see if that gets them to go away
posted by taquito sunrise at 10:34 AM on April 23, 2022


I feel like pretending you didn't understand someone is a more specific situation than making someone aware you're paying attention to their speech, & I'm not sure how meaningfully you can generalize their reaction, especially when it comes to enunciating syllables & pronouncing consonants

I know there's a replication crisis on and we should all be skeptical of everything, but FWIW, this particular result has replicated really well over the past 50 years — my guess would be dozens if not hundreds of times — in lots of populations, and with lots of different techniques for making-people-aware-of-your-attention. Fun story notwithstanding, this isn't a single cute experiment that people got carried away with. You can't do any kind of sociolinguistics without understanding whether the person you're studying is "correcting" their language, so figuring out what makes people more or less likely to do that is really fundamental to the whole field, and a lot of work has been done hammering out the details.
posted by nebulawindphone at 12:47 PM on April 23, 2022 [4 favorites]


I work for a psychologist whose research has actively made things better for people with chronic health conditions and disabilities. This is also the real world impact of psychology.

Unfortunately, that's also what the psychologists who have harmed a lot of people claim. They are certain that they are helping people. They are certain they are helping people with chronic health conditions, even when the patients are telling them they are causing harm.

That's kind of the problem: the standards of evidence are generally not good enough to tell if something is true or not or if an intervention is helping or not. As a field, psychology doesn't seem to be able to separate the good from the bad or deal with researchers who conduct badly flawed studies. I have no doubt there are positive impacts in some cases, but the evidence to separate helpful from harmful or helpful from placebo isn't generally there, in my experience.
posted by ssg at 1:19 PM on April 23, 2022 [2 favorites]


Methodology, folks. This is all about methodology.

There is currently a very hard push in the psych world for unblinded subjective outcome measures to be considered at least sufficient, if not the equivalent to properly blinded and/or objective outcome measures.

Anybody who doesn't realise how insane and dangerous that is hasn't understood the problem.

*cough*functional neurological disorders*cough*
posted by Pouteria at 2:46 PM on April 23, 2022 [1 favorite]


I know there's a replication crisis on and we should all be skeptical of everything, but FWIW, this particular result has replicated really well over the past 50 years — my guess would be dozens if not hundreds of times — in lots of populations, and with lots of different techniques for making-people-aware-of-your-attention.

that's completely fair, yeah

I apologize if what I said came across as skeptical of or trying to dispute the general conclusion that people change their speech when they're being observed; not arguing against that at all

the bit I quoted just really stuck out to me as "wait a minute, with this exact methodology there's a
different social phenomenon being invoked really strongly & unless you're doing something to control for that I don't think this particular study proves specifically that people change their speech merely from knowing it's being observed"

I'll also mention that I'm not a scientist, just a random autistic person who gets their "wait this thing on the internet is a MISTAKE!" switch triggered occasionally, & I went directly to the comment box after reading the quoted bit which triggered that

so if there's a larger context in which social scientists are taking sides & arguing them in this thread, I'm not attempting to take any of those sides or argue any of those arguments, just going to bat for one highly specific thing which is my M.O. until I die I guess
posted by taquito sunrise at 3:15 PM on April 23, 2022 [4 favorites]


Oh yeah totally, that's a reasonable MO. FWIW, my subtext is I'm a linguist who's grumpy that a thread about a cool linguist got turned into a fight about replication problems in a totally different field, but idk if anyone else is having that subtext conversation or if it's just me being grumpy into the void.
posted by nebulawindphone at 7:47 PM on April 23, 2022 [5 favorites]


Mod note: Yes, just to nudge this a little bit: I think the question of Should Psychology Even Exist As A Practice / Field is an enormous, rambling topic that has no limiting focus, and that the little known Labov paper (from a noted linguist addressing a group of teachers, about children being tested for verbal skills ) that actually is ultimately the main subject here is very interesting indeed, and worth discussion, but we're having a bit of a bifurcation in the thread. Here is the tweet that instigated the Labov find.
posted by taz (staff) at 12:34 AM on April 24, 2022 [6 favorites]


Apologies, Taz, and all, if the way I framed the initial post lead to a derail. I suspect choosing to bring up experiments with a much more complicated history was a mistake. I was delighted by the paper and surprised by the publication date. (It's also in a field I know almost nothing about. Perhaps one filled with mines I had not entirely considered.)
posted by eotvos at 2:30 PM on April 24, 2022 [1 favorite]


« Older Mirya is gone   |   It's not even a baseball game. It's a circus. Newer »


This thread has been archived and is closed to new comments