Why we can't have nice research reporting
July 17, 2012 8:41 AM Subscribe
Just because you don't like a study doesn't mean it's wrong. Gawker takes the rest of the blog world to task for misinterpreting this new paper on women who watch televised sports.
More research from the lead author.
More research from the lead author.
This is like a tempest in a recursive teapot.
A critique of a blogstorm over a misinterpretation of a communications study in a low impact qualitative journal.
posted by srboisvert at 8:56 AM on July 17, 2012 [6 favorites]
A critique of a blogstorm over a misinterpretation of a communications study in a low impact qualitative journal.
posted by srboisvert at 8:56 AM on July 17, 2012 [6 favorites]
I read a lot of feminist blogs and had not heard of this article before, so I don't think that Gawker can possibly be taking to task "the rest of the blog world." I also dispute the implication that Jezebel is or has ever claimed to be a feminist blog.
It seems like an okay study to me, not great, but with some interesting info. I agree with the criticism that it's hard to say anything about "women" with a sample size of 19.
It was also surprising to me that they seem to have specifically selected only women who were married to men. Since "women" in general are not all married to men, it seems that one should probably not be drawing conclusions about "women" from this study. Single women might have more time to watch the sports they want to watch, rather than those chosen by male partners. Certainly, the biggest women's professional sports fans I know are lesbians, which I wouldn't imagine is a shocking fact.
posted by hydropsyche at 9:00 AM on July 17, 2012 [1 favorite]
It seems like an okay study to me, not great, but with some interesting info. I agree with the criticism that it's hard to say anything about "women" with a sample size of 19.
It was also surprising to me that they seem to have specifically selected only women who were married to men. Since "women" in general are not all married to men, it seems that one should probably not be drawing conclusions about "women" from this study. Single women might have more time to watch the sports they want to watch, rather than those chosen by male partners. Certainly, the biggest women's professional sports fans I know are lesbians, which I wouldn't imagine is a shocking fact.
posted by hydropsyche at 9:00 AM on July 17, 2012 [1 favorite]
It is interesting to me how often when social science research is presented it is either mocked as being too obviously right to have bothered looking at all (i.e. the study confirms our intuitions) or mocked as being too obviously wrong to believe (i.e. the study disconfirms our intuitions).
In this case, it was a "to obviously right" paper (Some people like watching sports as a way to connect with their family - I love watching football with my Dad and the Red Sox with my Mom, for instance - and it so happens women are people, too, and husbands count as family) being reported by a confused press as a "too obviously wrong" paper (Women watch sports because their husband likes sports.) Hilarity ensues.
posted by Slap*Happy at 9:05 AM on July 17, 2012 [2 favorites]
In this case, it was a "to obviously right" paper (Some people like watching sports as a way to connect with their family - I love watching football with my Dad and the Red Sox with my Mom, for instance - and it so happens women are people, too, and husbands count as family) being reported by a confused press as a "too obviously wrong" paper (Women watch sports because their husband likes sports.) Hilarity ensues.
posted by Slap*Happy at 9:05 AM on July 17, 2012 [2 favorites]
I should be clear that I read the article and that they did address the heterosexuality thing in the Discussion, but otherwise they talked about these 19 women as representaive of "women". I am a natural, not social, scientist, and it drives me crazy when I see lazy generalities of this sort in papers in my discipline as well.
posted by hydropsyche at 9:06 AM on July 17, 2012 [1 favorite]
posted by hydropsyche at 9:06 AM on July 17, 2012 [1 favorite]
I feel like the real problem with this paper is it's seemingly unquestioned goal of encouraging women to play and particularly watch women play what have traditionally been male athletic activities. The author says:
Just because someone said you couldn't/shouldn't do something, doesn't mean you actually wanted to do it in the first place
posted by crayz at 9:08 AM on July 17, 2012 [7 favorites]
Although Title IX has opened doors for sport participation—and the women we interviewed have been beneficiaries—the difference between the images in a WNBA game or a Ladies Professional Golf Association (LPGA) tournament and the everyday lives of these women are a significant barrier in making these media texts popular. We speculate that the women we interviewed could not see themselves—or their ideal (gendered) selves—in the airing of such a competitive women’s event. Images of professional female athletes playing basketball, football, or soccer do not mesh with participants’ domestic reality or socially cultivated aspirations; in fact, the "shocking difference" (Fiske, 1989, p. 104) between the images of powerful female athletes—rejecting traditional gender roles—and the lives of the women we interviewed creates a very large gap.The fallacy here seems to be the presumption that feminism/equality means "doing the stuff men do/used to do". The fact that men give themselves trauma-induced brain damage by running into each other wearing metal helmets, and other men give themselves alcohol-induced brain damage while passively watching the spectacle from a sofa, and the whole phenomena mostly excludes women... maybe women should just let men destroy themselves? Or try to build a society in which all people spend their lives on more positive activities, rather than a society where all people do the idiotic things the ape-men dreamed up for themselves?
Just because someone said you couldn't/shouldn't do something, doesn't mean you actually wanted to do it in the first place
posted by crayz at 9:08 AM on July 17, 2012 [7 favorites]
Was it really possible that a 21-page double-blind, peer-reviewed academic study with 80 separate cited sources could be so TOTALLY BULLSHIT that a typical non-Ph.D. non-scientist writing on a blog could dismiss it out of hand simply by considering what they believed its conclusions to be?
Whether or not this particular study has its flaws and limitations, or is actually important to anyone, we need more blog posts like this one.
posted by gurple at 9:09 AM on July 17, 2012 [5 favorites]
Whether or not this particular study has its flaws and limitations, or is actually important to anyone, we need more blog posts like this one.
posted by gurple at 9:09 AM on July 17, 2012 [5 favorites]
Wait, was the study particularly about watching Football? I kind of assumed that it included all sports. Or maybe women's golf and basketball is WAY more hardcore than I thought.
Let's maybe leave the issues with football to another thread. There plenty of other sports that don't have those types of issues.
posted by VTX at 9:15 AM on July 17, 2012 [1 favorite]
Let's maybe leave the issues with football to another thread. There plenty of other sports that don't have those types of issues.
posted by VTX at 9:15 AM on July 17, 2012 [1 favorite]
A critique of a blogstorm over a misinterpretation of a communications study in a low impact qualitative journal.
Most social science and/or qualitative journals have low-impact factors. This does not mean that they are not good journals. What it means is that there are simply many, many fewer people working and publishing (and therefore citing each other) than there are in cell biology. I just worked on a CV for a very senior, influential social scientist - someone doing lots of influential research. I don't think she had a publication in a journal with an impact factor higher than 10 -- for comparison, Nature has an impact factor of 36.280. Of course, the New England Journal of Medicine has an impact factor of 53.48. Does this mean that it is substantially better than Nature, or that medical articles are just so much more likely to be cited than pure science ones?
One of the very best journals in my field, the Economic History Review, has an impact factor of 0.781. Past and Present is another excellent journal, and has an impact factor of 0.247.
Moral of the Story: impact factors only really matter to physical scientists and clinical researchers.
posted by jb at 9:28 AM on July 17, 2012 [11 favorites]
Most social science and/or qualitative journals have low-impact factors. This does not mean that they are not good journals. What it means is that there are simply many, many fewer people working and publishing (and therefore citing each other) than there are in cell biology. I just worked on a CV for a very senior, influential social scientist - someone doing lots of influential research. I don't think she had a publication in a journal with an impact factor higher than 10 -- for comparison, Nature has an impact factor of 36.280. Of course, the New England Journal of Medicine has an impact factor of 53.48. Does this mean that it is substantially better than Nature, or that medical articles are just so much more likely to be cited than pure science ones?
One of the very best journals in my field, the Economic History Review, has an impact factor of 0.781. Past and Present is another excellent journal, and has an impact factor of 0.247.
Moral of the Story: impact factors only really matter to physical scientists and clinical researchers.
posted by jb at 9:28 AM on July 17, 2012 [11 favorites]
I have to have tenure in social sciences now to ask "Isn't 19 subjects a rather small sample size?"
posted by Sidhedevil at 9:28 AM on July 17, 2012
posted by Sidhedevil at 9:28 AM on July 17, 2012
here's an impact factor from a more cohesive field than history -- the American Sociological Review (one of the top sociology journals) - impact factor of 4.422.
As for the sample size: I recently listened in on a discussion about group sizes for focus group research. Focus group research is inherently exploratory: you aren't testing questions with a population-based sample, you are trying to figure out what questions you should be asking. (A lot of survey-based research has a focus group as it's pilot/planning stage). The researchers were saying that at about 20-30 participants from similar backgrounds, you get a "saturation of themes" - people saying that same stuff that other people have already said. If you want to compare two groups of people - for example, a group with one chronic condition versus healthy controls or another condition, you aim for about 20-30 participants per comparison group -- broken into actual focus groups of 6-8 people (better discussion that way).
So 19 for their study is maybe on the low side, but not unreasonable.
posted by jb at 9:34 AM on July 17, 2012 [3 favorites]
As for the sample size: I recently listened in on a discussion about group sizes for focus group research. Focus group research is inherently exploratory: you aren't testing questions with a population-based sample, you are trying to figure out what questions you should be asking. (A lot of survey-based research has a focus group as it's pilot/planning stage). The researchers were saying that at about 20-30 participants from similar backgrounds, you get a "saturation of themes" - people saying that same stuff that other people have already said. If you want to compare two groups of people - for example, a group with one chronic condition versus healthy controls or another condition, you aim for about 20-30 participants per comparison group -- broken into actual focus groups of 6-8 people (better discussion that way).
So 19 for their study is maybe on the low side, but not unreasonable.
posted by jb at 9:34 AM on July 17, 2012 [3 favorites]
Anyways - none of the researchers in the said discussion had tenure. But they had, you know, studied focus group techniques and based their ideas of a good focus group size on the decades of experience that social scientists have with focus groups.
posted by jb at 9:36 AM on July 17, 2012
posted by jb at 9:36 AM on July 17, 2012
"I have to have tenure in social sciences now to ask "Isn't 19 subjects a rather small sample size?""
"but otherwise they talked about these 19 women as representaive of "women". I am a natural, not social, scientist, and it drives me crazy when I see lazy generalities of this sort in papers in my discipline as well."
FROM THE ARTICLE
"As to the much-criticized fact that the study used a relatively small number of women, Whiteside explained: "This is a qualitative project — hence group interviews with 19 women— as opposed to a wide-scale survey using a random sample. The strength of such an approach is it allows a researcher to go beyond describing a given social phenomena (e.g. women's sports lack a fan base) to explaining it. In fact, research that begins from the standpoint and everyday experiences of women is often considered a feminist approach to research (as opposed to male-centric modes 'discovery') because it privileges women's voices and experiences in the production of knowledge." Whiteside says that while standard numerical quantitative studies are based on set categories of identity (age, gender, etc.), "Our study thinks about how those identity categories are created... I would argue that identity categories are inherently restrictive and a product of power relations; once we assign an identity to a person, there is an expectation toward behavior. This is admittedly an abstract concept and difficult to report in a 300-word story, especially in a world in which measurable, objectively observed data is considered most worthy. This latter point reveals how epistemology (philosophy about what counts as knowledge) may figure into women's oppression. Although feminists hedge from assigning one method as THE feminist method, many would argue that qualitative research in the form of interviews, etc. can be feminist in that it situates the production of knowledge in women's everyday experiences and in their own voices."
. . . It is useful, when we decide to offhandedly offer our uninformed analysis about a study, to reflect upon the fact that the author of the study has probably already considered all of these objections—which are, after all, usually little more than the first knee-jerk reaction that pops into our heads. Whiteside, for example, is a Ph.D and an academic and is of course aware that it is impossible to generalize from 19 women to the entire world. "Applying any research 'finding' to a wider population is only* appropriate when the study draws from a random sample that is sufficiently representative of that wider population– which we did not do, nor try to do," she says. "But, again, our goal was not to describe a social trend at a meta-level but to interpret it, something for which qualitative research provides an especially useful toolkit." "
If you all are going to offer criticisms, at least have the courtesy to acknowledge the essay's points.
posted by oddman at 9:47 AM on July 17, 2012 [4 favorites]
"but otherwise they talked about these 19 women as representaive of "women". I am a natural, not social, scientist, and it drives me crazy when I see lazy generalities of this sort in papers in my discipline as well."
FROM THE ARTICLE
"As to the much-criticized fact that the study used a relatively small number of women, Whiteside explained: "This is a qualitative project — hence group interviews with 19 women— as opposed to a wide-scale survey using a random sample. The strength of such an approach is it allows a researcher to go beyond describing a given social phenomena (e.g. women's sports lack a fan base) to explaining it. In fact, research that begins from the standpoint and everyday experiences of women is often considered a feminist approach to research (as opposed to male-centric modes 'discovery') because it privileges women's voices and experiences in the production of knowledge." Whiteside says that while standard numerical quantitative studies are based on set categories of identity (age, gender, etc.), "Our study thinks about how those identity categories are created... I would argue that identity categories are inherently restrictive and a product of power relations; once we assign an identity to a person, there is an expectation toward behavior. This is admittedly an abstract concept and difficult to report in a 300-word story, especially in a world in which measurable, objectively observed data is considered most worthy. This latter point reveals how epistemology (philosophy about what counts as knowledge) may figure into women's oppression. Although feminists hedge from assigning one method as THE feminist method, many would argue that qualitative research in the form of interviews, etc. can be feminist in that it situates the production of knowledge in women's everyday experiences and in their own voices."
. . . It is useful, when we decide to offhandedly offer our uninformed analysis about a study, to reflect upon the fact that the author of the study has probably already considered all of these objections—which are, after all, usually little more than the first knee-jerk reaction that pops into our heads. Whiteside, for example, is a Ph.D and an academic and is of course aware that it is impossible to generalize from 19 women to the entire world. "Applying any research 'finding' to a wider population is only* appropriate when the study draws from a random sample that is sufficiently representative of that wider population– which we did not do, nor try to do," she says. "But, again, our goal was not to describe a social trend at a meta-level but to interpret it, something for which qualitative research provides an especially useful toolkit." "
If you all are going to offer criticisms, at least have the courtesy to acknowledge the essay's points.
posted by oddman at 9:47 AM on July 17, 2012 [4 favorites]
I acknowledge the articles' points and its limitations--I don't think Gawker or the wording of the FPP do.
posted by hydropsyche at 9:49 AM on July 17, 2012 [2 favorites]
posted by hydropsyche at 9:49 AM on July 17, 2012 [2 favorites]
A couple of points:
- I agree with the Gawker writer that it's ridiculous for blogs to criticize a study without truly taking the time to understand it. However, I don't understand why he focuses so much on the "feminist blogs", while barely mentioning the equally lazy/inaccurate reporting from the mainstream media (LA Times headline).
- I'm a woman, a feminist, and a sports fan, and I normally do not choose to watch women's sports. Why? Because I like watching the best athletes in a given field - and apart from specific sports such as gymnastics (which I'm not all that interested in), the best athletes are always going to be the men. I find NHL hockey a lot more interesting than women's hockey. For the same reason, I love NFL football but choose not to watch CFL football unless nothing else is on. I would assume many sports fans feel the same way.
posted by barnoley at 10:04 AM on July 17, 2012 [1 favorite]
- I agree with the Gawker writer that it's ridiculous for blogs to criticize a study without truly taking the time to understand it. However, I don't understand why he focuses so much on the "feminist blogs", while barely mentioning the equally lazy/inaccurate reporting from the mainstream media (LA Times headline).
- I'm a woman, a feminist, and a sports fan, and I normally do not choose to watch women's sports. Why? Because I like watching the best athletes in a given field - and apart from specific sports such as gymnastics (which I'm not all that interested in), the best athletes are always going to be the men. I find NHL hockey a lot more interesting than women's hockey. For the same reason, I love NFL football but choose not to watch CFL football unless nothing else is on. I would assume many sports fans feel the same way.
posted by barnoley at 10:04 AM on July 17, 2012 [1 favorite]
From the Gawker piece: Someone who writes news stories or blog posts about scientific studies is not required to have a Ph.D. in every scientific field.
I think I just figured out what to do with the glut of unemployed PhDs I keep hearing about on the Blue....
posted by Panjandrum at 10:22 AM on July 17, 2012 [3 favorites]
I think I just figured out what to do with the glut of unemployed PhDs I keep hearing about on the Blue....
posted by Panjandrum at 10:22 AM on July 17, 2012 [3 favorites]
I have to have tenure in social sciences now to ask "Isn't 19 subjects a rather small sample size?"
Not necessarily, but you have to keep in mind how sampling plays different roles in qualitative vs. quantitative research. In both qualitative and quantitative research, the goal is to find a "representative" sample (i.e., a smaller group that "represents" the larger group it came from).
In quantitative research, the sample is representative to the extent that the process of selecting members of the sample is random. With a random sample, you can calculate descriptive statistics and your margin of error for those statistics. Generally speaking, for quantitative samples, the margin of error decreases as the absolute size of the sample increases. In other words, bigger is usually better when it comes to quantitative research. Most methodological choices made by researchers focus on how much it costs in time and money to gather data from each additional member of the sample vs. how much they can improve accuracy and reduce their margin of error.
In qualitative research, the role of sampling is completely different, because bigger is not necessarily better. Because qualitative samples are chosen nonrandomly, increasing the sample size does not do anything to increase the accuracy of the sample. Instead, qualitative samples are representative, because the members of those samples are "representative" of types and categories relevant to proving or disproving the theories that inspired the research in the first place. (Alternatively, a qualitative sample may be used inductively to generate a theory about a causal process that hasn't been theorized about before.)
How does this apply to the current study? The researchers start with an extremely interesting research question, "If participation in women's sports has exploded because of Title IX, why is there such a limited media audience for women's sports, when compared to men's sports? Shouldn't the TV audience for women's sports be increasing, not in the basement? Shouldn't WNBA be expanding, not folding up teams?" In order to answer this question, the researchers selected 19 women who accompanied their husbands to a professional conference who also happened to be sports fans. Why limit themselves to heterosexual married women? First, as the Discussion section makes clear, there is already a lot of research about lesbians and how they consume women's sports, which the researchers want to avoid duplicating. Second, the research question is focused on why women as a mass audience fail to consume women's sports. Since married heterosexual women are considered more "representative" of the women's mass audience than lesbians (and also more desired by advertisers), women from this group are more relevant to answering the research question posed by this study.
So what did the study find?
1. Married heterosexual women who consume sports media are too busy with chores and housework to consume most sports broadcasts except in a random, piecemeal fashion that does not fit with the schedule of a standard athletic team. If I were in sports marketing, this finding would be very relevant, because it would give me some clues on how to structure sports media to appeal to female consumers.
2. Married heterosexual women spend less time consuming sports TV and media that they enjoy personally than getting socially involved in the consumption of the sports TV and media enjoyed by the men in their lives. In practical terms, even if women had twice as much interest in women's sports as they do now, they are not necessarily going to get the opportunity to watch it, because the male sports fans in their lives are probably not going to relinquish the remote to watch women's basketball. This is consistent with a lot of media studies that emphasize how TV viewing is not the result of rational individuals making decisions about the content that maximizes their enjoyment, but how TV viewing is more of a social process.
3. The women in this study were themselves very ambivalent about women's sports. Some women found women's sports boring, whereas other participants might find female athletes difficult for them to relate to as married women, because female athletes are more likely to be gender nonconformist than the average married woman.
4. Previous research cited by the researchers speculated that women who watch sports might identify with the "male gaze," becoming more masculine in their viewing habits. The researchers argue that this is not true, because the women in their focus groups were concerned about lack of equity in the treatment of women's sports and the sexualization of women's athletes. The cheesecake photo shoots of race car driver Danica Patrick were specifically mentioned by the women in the study.
Now why would you want to address these questions with a qualitative sample instead of a larger quantitative one? Part of the reason is that a quantitative survey will only succeed if you know the right questions to ask beforehand. With a qualitative study, you have more room to be exploratory and keep an open mind about what causal factors are most important, much more so than with a quantitative survey. In addition, the percentage of potential female sports fans might be too small a percentage of the audience to survey efficiently with a random sample. Among the 19 women in the study, only one woman was an unapologetic, unambivalent booster of women's sports (in this case, she was a frequent attender of WNBA games). So, if that's the case, you can expect only about 5% of women in a random survey (if you're lucky) to be unapologetic women's sports fans. In other words, if you wanted to gather randomized data about women's sports fans, you might be throwing away 95% of your sample, which is not only inefficient, but may give you less insight than a qualitative study.
Now, if you've read the article, you may ask why they didn't go into more depth justifying focus groups with 19 women as a data-gathering method. The reason is that it's a qualitative journal, not the Journal of Random Stuff Appealing to Bloggers and Mefites. The journal does not have to do elaborate self-justifications of qualitative every single time, because their audience is not a bunch of Internet commenters, some of whom have to be convinced that qualitative research has any legitimacy at all.
Finally, if you haven't read the article, what's wrong with you? I see these threads all the time, but most of the responses probably come from the TL;DR crowd than the RTFA crowd. All of this dynamic makes journalistic coverage of social science research a neverending cycle of Gresham's Law where bad coverage that gives you clickbait drives out the good coverage.
First, the article gets written and published in a peer-reviewed journal. Second, some media relations person at some university who has no academic training in the field writes a press release about the article that completely simplifies or bungles the main points of the study. Third, some journalist writes an article about the study based on the media relations department's press release, while running roughshod over any nuance in the original article. Finally, bloggers take the journalist's article and critique the study based on their misunderstanding of the journalist's article as filtered through their own social and political biases. The result is a smudgy Xerox of a Xerox of a Xerox of a document that has almost no resemblance to the original. And thus the cycle of Gresham's Law of Social Science Research continues.
posted by jonp72 at 10:40 AM on July 17, 2012 [23 favorites]
Not necessarily, but you have to keep in mind how sampling plays different roles in qualitative vs. quantitative research. In both qualitative and quantitative research, the goal is to find a "representative" sample (i.e., a smaller group that "represents" the larger group it came from).
In quantitative research, the sample is representative to the extent that the process of selecting members of the sample is random. With a random sample, you can calculate descriptive statistics and your margin of error for those statistics. Generally speaking, for quantitative samples, the margin of error decreases as the absolute size of the sample increases. In other words, bigger is usually better when it comes to quantitative research. Most methodological choices made by researchers focus on how much it costs in time and money to gather data from each additional member of the sample vs. how much they can improve accuracy and reduce their margin of error.
In qualitative research, the role of sampling is completely different, because bigger is not necessarily better. Because qualitative samples are chosen nonrandomly, increasing the sample size does not do anything to increase the accuracy of the sample. Instead, qualitative samples are representative, because the members of those samples are "representative" of types and categories relevant to proving or disproving the theories that inspired the research in the first place. (Alternatively, a qualitative sample may be used inductively to generate a theory about a causal process that hasn't been theorized about before.)
How does this apply to the current study? The researchers start with an extremely interesting research question, "If participation in women's sports has exploded because of Title IX, why is there such a limited media audience for women's sports, when compared to men's sports? Shouldn't the TV audience for women's sports be increasing, not in the basement? Shouldn't WNBA be expanding, not folding up teams?" In order to answer this question, the researchers selected 19 women who accompanied their husbands to a professional conference who also happened to be sports fans. Why limit themselves to heterosexual married women? First, as the Discussion section makes clear, there is already a lot of research about lesbians and how they consume women's sports, which the researchers want to avoid duplicating. Second, the research question is focused on why women as a mass audience fail to consume women's sports. Since married heterosexual women are considered more "representative" of the women's mass audience than lesbians (and also more desired by advertisers), women from this group are more relevant to answering the research question posed by this study.
So what did the study find?
1. Married heterosexual women who consume sports media are too busy with chores and housework to consume most sports broadcasts except in a random, piecemeal fashion that does not fit with the schedule of a standard athletic team. If I were in sports marketing, this finding would be very relevant, because it would give me some clues on how to structure sports media to appeal to female consumers.
2. Married heterosexual women spend less time consuming sports TV and media that they enjoy personally than getting socially involved in the consumption of the sports TV and media enjoyed by the men in their lives. In practical terms, even if women had twice as much interest in women's sports as they do now, they are not necessarily going to get the opportunity to watch it, because the male sports fans in their lives are probably not going to relinquish the remote to watch women's basketball. This is consistent with a lot of media studies that emphasize how TV viewing is not the result of rational individuals making decisions about the content that maximizes their enjoyment, but how TV viewing is more of a social process.
3. The women in this study were themselves very ambivalent about women's sports. Some women found women's sports boring, whereas other participants might find female athletes difficult for them to relate to as married women, because female athletes are more likely to be gender nonconformist than the average married woman.
4. Previous research cited by the researchers speculated that women who watch sports might identify with the "male gaze," becoming more masculine in their viewing habits. The researchers argue that this is not true, because the women in their focus groups were concerned about lack of equity in the treatment of women's sports and the sexualization of women's athletes. The cheesecake photo shoots of race car driver Danica Patrick were specifically mentioned by the women in the study.
Now why would you want to address these questions with a qualitative sample instead of a larger quantitative one? Part of the reason is that a quantitative survey will only succeed if you know the right questions to ask beforehand. With a qualitative study, you have more room to be exploratory and keep an open mind about what causal factors are most important, much more so than with a quantitative survey. In addition, the percentage of potential female sports fans might be too small a percentage of the audience to survey efficiently with a random sample. Among the 19 women in the study, only one woman was an unapologetic, unambivalent booster of women's sports (in this case, she was a frequent attender of WNBA games). So, if that's the case, you can expect only about 5% of women in a random survey (if you're lucky) to be unapologetic women's sports fans. In other words, if you wanted to gather randomized data about women's sports fans, you might be throwing away 95% of your sample, which is not only inefficient, but may give you less insight than a qualitative study.
Now, if you've read the article, you may ask why they didn't go into more depth justifying focus groups with 19 women as a data-gathering method. The reason is that it's a qualitative journal, not the Journal of Random Stuff Appealing to Bloggers and Mefites. The journal does not have to do elaborate self-justifications of qualitative every single time, because their audience is not a bunch of Internet commenters, some of whom have to be convinced that qualitative research has any legitimacy at all.
Finally, if you haven't read the article, what's wrong with you? I see these threads all the time, but most of the responses probably come from the TL;DR crowd than the RTFA crowd. All of this dynamic makes journalistic coverage of social science research a neverending cycle of Gresham's Law where bad coverage that gives you clickbait drives out the good coverage.
First, the article gets written and published in a peer-reviewed journal. Second, some media relations person at some university who has no academic training in the field writes a press release about the article that completely simplifies or bungles the main points of the study. Third, some journalist writes an article about the study based on the media relations department's press release, while running roughshod over any nuance in the original article. Finally, bloggers take the journalist's article and critique the study based on their misunderstanding of the journalist's article as filtered through their own social and political biases. The result is a smudgy Xerox of a Xerox of a Xerox of a document that has almost no resemblance to the original. And thus the cycle of Gresham's Law of Social Science Research continues.
posted by jonp72 at 10:40 AM on July 17, 2012 [23 favorites]
hydropsyche : It was also surprising to me that they seem to have specifically selected only women who were married to menFrom the paper:
We also looked for women who were likely to be middle-class, married mothers with children at home, who represent a viewing group sought after by sports programmers.I'm guessing sports programmers means sports advertisers, and that is how this paper got funded.
posted by falcon at 12:00 PM on July 17, 2012
I was going to post a jokey comment about a checklist of how to refute a study you don't like, with stuff like "the sample size is too small" and "the journal it's in doesn't matter" and "I bet [soandso] funded this study!", but I got beat to the punch by actual comments.
posted by 0xFCAF at 12:03 PM on July 17, 2012 [6 favorites]
posted by 0xFCAF at 12:03 PM on July 17, 2012 [6 favorites]
this all reminds me of the time I complained to peter sagal about their usage of the ridiculous louanne brizendine fictum about women using 352.4% more words than men to say the same thing.
Peter's response: "Sometimes a story is too good to fact-check."
posted by lodurr at 12:13 PM on July 17, 2012 [1 favorite]
Peter's response: "Sometimes a story is too good to fact-check."
posted by lodurr at 12:13 PM on July 17, 2012 [1 favorite]
I actually do have some understanding of qualitative research, albeit second-hand, as someone who wrote reports to funders on a multi-million-dollar five-year qualitative research project. I certainly don't know more about this than the sociologists on this thread can possibly imagine, but I know considerably more than jack point shit.
posted by Sidhedevil at 12:14 PM on July 17, 2012 [1 favorite]
posted by Sidhedevil at 12:14 PM on July 17, 2012 [1 favorite]
I'm guessing sports programmers means sports advertisers.
"Sports programmers" would mean people who sell time to advertisers, which is effectively the same for your point.
posted by lodurr at 12:14 PM on July 17, 2012
"Sports programmers" would mean people who sell time to advertisers, which is effectively the same for your point.
posted by lodurr at 12:14 PM on July 17, 2012
Also I don't care one way or another about the study. A nineteen-person focus group seems overly small from my experience. jb says that 20-30 is considered optimal, so I bow to that. The projects I am most familiar with used 30 or more, but those were all in the education sector.
posted by Sidhedevil at 12:18 PM on July 17, 2012
posted by Sidhedevil at 12:18 PM on July 17, 2012
I did a double case study with an n of 2 for my dissertation, and it was a darn good dissertation (it passed with distinction from Big Time U). That doesn't mean you could draw any definitive answers from it, but it raised some darn good questions and it earned me a Ph.D., by gum. I am willing to bet no one has ever read it since.
Anyone who reads any research at all should read it like an academic: with a skeptical eye, close attention to the bibliography, and phasers set on stun. That lack of that suspicion and awareness on the part of the media is what makes people so exasperated with science.
However, if I read the article carefully, I understand why I watch so much sports and why I'm a competitive athlete, even though my husband hates sports.
posted by Peach at 12:43 PM on July 17, 2012 [2 favorites]
Anyone who reads any research at all should read it like an academic: with a skeptical eye, close attention to the bibliography, and phasers set on stun. That lack of that suspicion and awareness on the part of the media is what makes people so exasperated with science.
However, if I read the article carefully, I understand why I watch so much sports and why I'm a competitive athlete, even though my husband hates sports.
posted by Peach at 12:43 PM on July 17, 2012 [2 favorites]
barnoley: Because I like watching the best athletes in a given field - and apart from specific sports such as gymnastics (which I'm not all that interested in), the best athletes are always going to be the men.
Even if we accept the premise that the best athletes are always going to be men, it does not necessarily follow that the most enjoyable games to watch are going to be between the best athletes.
Case in point, the 2011 Women's World Cup (soccer) had an absolute stone cold classic final. It's one of the most gripping games of football I've ever watched, and certainly the best final of a major international tournament this century, male or female. I've watched most of them, and this is by far the standout game.
posted by Kattullus at 12:57 PM on July 17, 2012 [1 favorite]
Even if we accept the premise that the best athletes are always going to be men, it does not necessarily follow that the most enjoyable games to watch are going to be between the best athletes.
Case in point, the 2011 Women's World Cup (soccer) had an absolute stone cold classic final. It's one of the most gripping games of football I've ever watched, and certainly the best final of a major international tournament this century, male or female. I've watched most of them, and this is by far the standout game.
posted by Kattullus at 12:57 PM on July 17, 2012 [1 favorite]
men give themselves trauma-induced brain damage by running into each other wearing metal helmets
What sport are you talking about? Rollerball?
posted by msalt at 1:00 PM on July 17, 2012 [1 favorite]
What sport are you talking about? Rollerball?
posted by msalt at 1:00 PM on July 17, 2012 [1 favorite]
the premise that the best athletes will always be men is flawed in several regards -- not least being that the men's game often ends up emphasizing stuff that's been selected for by outside pressures like money, and de-emphasizes skills that were more critical before those pressures got ratcheted up to where they are.
Basketball, for example, and probably soccer too: a lot of the women are showing levels of proficiency in some skills that are not seen in the men's game anymore -- because with ever-stronger, larger and faster men in the game (a function of the money involved), they matter less.
So that makes the womens' games more interesting to a lot of aficionados. I know a number of die-hard old college hoops fans who swear they prefer to watch women's ball. The women are "better" by one way of judging skill -- "worse", of course, by objective measures like "could they beat the Lakers?"
posted by lodurr at 1:05 PM on July 17, 2012 [2 favorites]
Basketball, for example, and probably soccer too: a lot of the women are showing levels of proficiency in some skills that are not seen in the men's game anymore -- because with ever-stronger, larger and faster men in the game (a function of the money involved), they matter less.
So that makes the womens' games more interesting to a lot of aficionados. I know a number of die-hard old college hoops fans who swear they prefer to watch women's ball. The women are "better" by one way of judging skill -- "worse", of course, by objective measures like "could they beat the Lakers?"
posted by lodurr at 1:05 PM on July 17, 2012 [2 favorites]
For really good watching, I recommend women's rugby. Nothing like watching a woman shove a tampon up her nose to stop the bleeding to get your pulse racing.*
*You can tell I have actually watched some women's rugby up close.
posted by Peach at 1:08 PM on July 17, 2012 [2 favorites]
*You can tell I have actually watched some women's rugby up close.
posted by Peach at 1:08 PM on July 17, 2012 [2 favorites]
objective measures like "could they beat the Lakers?"
Heh, that's the measure most people care about in sports, I gather. If the Eagles lose while Mike Vick puts on an amazing show of skill I'm still screaming bloody murder at the TV. I do that because I care if they win or lose, and that is really what is lacking from a lot of women's sports. You don't need to be the best team in the world or have the best athletes, the world is full of 2nd tier soccer leagues with dedicated fans. It's about that personal investment and pride (those people who call the team "we" instead of "them") and women's sports are lagging behind in generating that. It's something that has to be cultivated over a long period of years. I started watching Eagles football with my Dad and still enjoy watching with him, bad headline writers read this as I only watch because of him, even though it's the only sport I invest serious money in and follow the team 365 days a year. I can't do that with another sport. It's just too much.
People like me drive the NFL fandom, it's not a rational hobby really and it takes up a lot of time. Women have their own interests already, and Men too...so any new hobby like following women's sports has a lot to compete with. I think a lot of this is just inertia.
People invest in women's figure skating, gymnastics, and other Olympic events but only part time. Full time pro-leagues just compete with too much other stuff. People invest in female athletes in motorsports and tennis (yes even beyond the pinup types) so it's not like the very idea of female professional athletes offends people. I think it's good there are studies like this out there because it is a solvable problem, it's just going to take time and study to figure out how to cultivate the fanatical fanbase.
More women should try and make it in motorsports. The same physical barriers to competing with men don't exist but you need a ton of skill, bravery, and toughness. It's high profile in various forms all over the world. More women at the top could change a lot of perceptions of female athletes. Just stay out of the way of the leader if you don't know what you're doing yet.
posted by furiousxgeorge at 2:11 PM on July 17, 2012 [1 favorite]
Heh, that's the measure most people care about in sports, I gather. If the Eagles lose while Mike Vick puts on an amazing show of skill I'm still screaming bloody murder at the TV. I do that because I care if they win or lose, and that is really what is lacking from a lot of women's sports. You don't need to be the best team in the world or have the best athletes, the world is full of 2nd tier soccer leagues with dedicated fans. It's about that personal investment and pride (those people who call the team "we" instead of "them") and women's sports are lagging behind in generating that. It's something that has to be cultivated over a long period of years. I started watching Eagles football with my Dad and still enjoy watching with him, bad headline writers read this as I only watch because of him, even though it's the only sport I invest serious money in and follow the team 365 days a year. I can't do that with another sport. It's just too much.
People like me drive the NFL fandom, it's not a rational hobby really and it takes up a lot of time. Women have their own interests already, and Men too...so any new hobby like following women's sports has a lot to compete with. I think a lot of this is just inertia.
People invest in women's figure skating, gymnastics, and other Olympic events but only part time. Full time pro-leagues just compete with too much other stuff. People invest in female athletes in motorsports and tennis (yes even beyond the pinup types) so it's not like the very idea of female professional athletes offends people. I think it's good there are studies like this out there because it is a solvable problem, it's just going to take time and study to figure out how to cultivate the fanatical fanbase.
More women should try and make it in motorsports. The same physical barriers to competing with men don't exist but you need a ton of skill, bravery, and toughness. It's high profile in various forms all over the world. More women at the top could change a lot of perceptions of female athletes. Just stay out of the way of the leader if you don't know what you're doing yet.
posted by furiousxgeorge at 2:11 PM on July 17, 2012 [1 favorite]
I don't really disagree with you -- I'm just saying "best athlete" isn't a simple assessment. (And I don't want to open up some godawful long thread about that 'cuz i gotta tell you, I'd bail fast, I'm just not that invested.)
posted by lodurr at 2:14 PM on July 17, 2012
posted by lodurr at 2:14 PM on July 17, 2012
the premise that the best athletes will always be men is flawed in several regards -- not least being that the men's game often ends up emphasizing stuff that's been selected for by outside pressures like money, and de-emphasizes skills that were more critical before those pressures got ratcheted up to where they are.
Basketball, for example, and probably soccer too: a lot of the women are showing levels of proficiency in some skills that are not seen in the men's game anymore -- because with ever-stronger, larger and faster men in the game (a function of the money involved), they matter less.
I actually do agree with you to some extent - I was just oversimplifying to make a point. (For example, I prefer watching women's tennis over men's tennis, for the reasons you note.) But the fact remains that men on average will always tend to be bigger and stronger than women on average. Which, in many sports that rely on strength and speed, will translate into a higher level of play.
posted by barnoley at 4:24 PM on July 17, 2012
Basketball, for example, and probably soccer too: a lot of the women are showing levels of proficiency in some skills that are not seen in the men's game anymore -- because with ever-stronger, larger and faster men in the game (a function of the money involved), they matter less.
I actually do agree with you to some extent - I was just oversimplifying to make a point. (For example, I prefer watching women's tennis over men's tennis, for the reasons you note.) But the fact remains that men on average will always tend to be bigger and stronger than women on average. Which, in many sports that rely on strength and speed, will translate into a higher level of play.
posted by barnoley at 4:24 PM on July 17, 2012
This basically boils down to 1) LAT wrote stupid "your move, feminists!" headline to provoke people to read their story, 2) feminist bloggers reacted to shitty reporting the way they would react to shitty evo psych reasoning and all the rest of the media's garbage, because at the end of the day it's hard to taxonomize all the shitty ways of cutting down women under the guise of "common sense."
The paper is interesting, and I originally clicked through and read what it really said, because I'm probably a lot like the women they interviewed and didn't react badly to it. What's actually unfortunate here is that "(mostly white) heterosexual married women" is equal to "women" as far as the reporting world is concerned, so that a survey on a small group of heterosexual married women becomes a conclusion about "women," for all intents and purposes at the LAT, and makes just about everyone feel steamrolled in the process. (Reporting on studies about "men" do the same damn thing, except they're usually justifying why men cheat, rape, or stare at women when they get all jaunty about it. Basically, it's pandering.)
posted by stoneandstar at 5:23 PM on July 17, 2012 [2 favorites]
The paper is interesting, and I originally clicked through and read what it really said, because I'm probably a lot like the women they interviewed and didn't react badly to it. What's actually unfortunate here is that "(mostly white) heterosexual married women" is equal to "women" as far as the reporting world is concerned, so that a survey on a small group of heterosexual married women becomes a conclusion about "women," for all intents and purposes at the LAT, and makes just about everyone feel steamrolled in the process. (Reporting on studies about "men" do the same damn thing, except they're usually justifying why men cheat, rape, or stare at women when they get all jaunty about it. Basically, it's pandering.)
posted by stoneandstar at 5:23 PM on July 17, 2012 [2 favorites]
Wait, though:
It is possible for scientific or academic research studies to be wrong. But they must be wrong for a reason. The fact that their conclusions just rub us the wrong way is not a reason. It is possible that our instincts have been proven wrong, by the science at hand.
... and then it continues to critique the knee-jerk response to bad newspaper reporting. But why include this? Is it a micro-lecture about feminism? Was a self-respecting feminist's first reaction supposed to be, "wait, maybe I am just watching sports for a man's sake?" I mean, we're talking about women's instincts about what women (like, themselves) do. You can argue from one standpoint that we know not what we do, or whatever, but if a study about women watching sports was saying "men watch sports like this, women watch sports like this," then I really think the valid point he makes is that we should be thinking, "wait, why is the LAT reporting this in a conspicuously bogus way, let me check the source material." Obviously these blogs have misapprehended the scope of the study, but this line seems to be something pulled from the same "nyah-nyah" part of discourse that the LAT and Gawker headlines were pulled from.
I mean, the title of the Gawker pieces as quoted in the FPP is just as shit-stirring as the LAT headline, imo.
Anyway, I have a hard time hating on the feminist blogs when the LAT article is so flagrant in the first place.
posted by stoneandstar at 5:27 PM on July 17, 2012 [1 favorite]
It is possible for scientific or academic research studies to be wrong. But they must be wrong for a reason. The fact that their conclusions just rub us the wrong way is not a reason. It is possible that our instincts have been proven wrong, by the science at hand.
... and then it continues to critique the knee-jerk response to bad newspaper reporting. But why include this? Is it a micro-lecture about feminism? Was a self-respecting feminist's first reaction supposed to be, "wait, maybe I am just watching sports for a man's sake?" I mean, we're talking about women's instincts about what women (like, themselves) do. You can argue from one standpoint that we know not what we do, or whatever, but if a study about women watching sports was saying "men watch sports like this, women watch sports like this," then I really think the valid point he makes is that we should be thinking, "wait, why is the LAT reporting this in a conspicuously bogus way, let me check the source material." Obviously these blogs have misapprehended the scope of the study, but this line seems to be something pulled from the same "nyah-nyah" part of discourse that the LAT and Gawker headlines were pulled from.
I mean, the title of the Gawker pieces as quoted in the FPP is just as shit-stirring as the LAT headline, imo.
Anyway, I have a hard time hating on the feminist blogs when the LAT article is so flagrant in the first place.
posted by stoneandstar at 5:27 PM on July 17, 2012 [1 favorite]
"3. The women in this study were themselves very ambivalent about women's sports. Some women found women's sports boring, whereas other participants might find female athletes difficult for them to relate to as married women, because female athletes are more likely to be gender nonconformist than the average married woman. "
This is an interesting point to me because one thing that got me really excited about women's sports was seeing the 1999 Women's World Cup (soccer). I've always enjoyed gymnastics (which is on TV a lot), and those women are astounding athletes, but they're all so tiny doing such impossible things while wearing makeup and glitter and smiling the whole time and most of them are teenagers. It's not really something I can relate to.
But when I saw the 1999 USA team in the Women's World Cup, those were women with hips and boobs and ponytails and in some cases toddlers. They sweated when they ran around and they got frizzy hair and they had concentration faces where they wrinkled their brows or looked pissed or whatever. Those were women LIKE ME, and it was so incredibly exciting because I honestly could not really recall seeing women like me playing sports, since televised women's elite-level sports was mostly gymnastics, figure skating, and diving. TV was all about the "pretty" women's sports. I got really into watching women's college basketball for much the same reason (although those women are alarmingly tall). But they come in a much broader variety of body types and their hair gets sweaty and frizzy during the game and that is GREAT. When I exercise MY hair gets sweaty and frizzy! (Stars! They're just like us!)
My husband and I watched the U.S. Olympic track trials a couple weeks ago (we like track & field when we happen to catch it on, usually by channel surfing, though we know nothing about the athletes and obviously aren't devoted fans) and we both agreed we had the warmest feelings towards the athletes who were hugging their toddlers after their races, because WE have toddlers! It's funny how something so small and irrelevant can make you identify with an athlete so much, but those are totally the guys we're cheering for at the Olympics, we decided.
Anyway, it really highlighted to me the importance of role models, and why programs where, for example, black professionals mentor young black students in impoverished inner-city schools are important -- it really, really helps to see that someone who is "like you" in some particular way can do the thing you want to do. All the women's sports I had ever seen were tiny, graceful girls doing amazing flipping-and-spinning sorts of things, and it's great to watch, but it isn't me. But when I saw the USA women's soccer team, it was just electrifying, because women just like me were playing amazing high-level soccer.
I don't know, it sounds silly, because intellectually I knew women could play any sport they wanted, but actually seeing these strong, athletic women on television kicking the shit out of soccer, sweating and frizzing and wearing sports bras, it was just so cool.
posted by Eyebrows McGee at 9:21 PM on July 17, 2012 [3 favorites]
This is an interesting point to me because one thing that got me really excited about women's sports was seeing the 1999 Women's World Cup (soccer). I've always enjoyed gymnastics (which is on TV a lot), and those women are astounding athletes, but they're all so tiny doing such impossible things while wearing makeup and glitter and smiling the whole time and most of them are teenagers. It's not really something I can relate to.
But when I saw the 1999 USA team in the Women's World Cup, those were women with hips and boobs and ponytails and in some cases toddlers. They sweated when they ran around and they got frizzy hair and they had concentration faces where they wrinkled their brows or looked pissed or whatever. Those were women LIKE ME, and it was so incredibly exciting because I honestly could not really recall seeing women like me playing sports, since televised women's elite-level sports was mostly gymnastics, figure skating, and diving. TV was all about the "pretty" women's sports. I got really into watching women's college basketball for much the same reason (although those women are alarmingly tall). But they come in a much broader variety of body types and their hair gets sweaty and frizzy during the game and that is GREAT. When I exercise MY hair gets sweaty and frizzy! (Stars! They're just like us!)
My husband and I watched the U.S. Olympic track trials a couple weeks ago (we like track & field when we happen to catch it on, usually by channel surfing, though we know nothing about the athletes and obviously aren't devoted fans) and we both agreed we had the warmest feelings towards the athletes who were hugging their toddlers after their races, because WE have toddlers! It's funny how something so small and irrelevant can make you identify with an athlete so much, but those are totally the guys we're cheering for at the Olympics, we decided.
Anyway, it really highlighted to me the importance of role models, and why programs where, for example, black professionals mentor young black students in impoverished inner-city schools are important -- it really, really helps to see that someone who is "like you" in some particular way can do the thing you want to do. All the women's sports I had ever seen were tiny, graceful girls doing amazing flipping-and-spinning sorts of things, and it's great to watch, but it isn't me. But when I saw the USA women's soccer team, it was just electrifying, because women just like me were playing amazing high-level soccer.
I don't know, it sounds silly, because intellectually I knew women could play any sport they wanted, but actually seeing these strong, athletic women on television kicking the shit out of soccer, sweating and frizzing and wearing sports bras, it was just so cool.
posted by Eyebrows McGee at 9:21 PM on July 17, 2012 [3 favorites]
« Older Monster Brains | If you want to live forever, then don't stop... Newer »
This thread has been archived and is closed to new comments
This is shocking. Shocking.
It is possible for scientific or academic research studies to be wrong. But they must be wrong for a reason. The fact that their conclusions just rub us the wrong way is not a reason. It is possible that our instincts have been proven wrong, by the science at hand.
It is interesting to me how often when social science research is presented it is either mocked as being too obviously right to have bothered looking at all (i.e. the study confirms our intuitions) or mocked as being too obviously wrong to believe (i.e. the study disconfirms our intuitions). But a big part of scientific research is carefully replacing intuition with observation: replacing mere belief with knowledge.
posted by Jonathan Livengood at 8:55 AM on July 17, 2012 [33 favorites]