Another look at counting the deaths in Iraq

November 12, 2006 7:46 PM

There's an interesting piece over at This American Life (titled "What's in number"). It touches on the previously discussed Lancet study and gives a better explanation of the methodology use. Be sure and check out Act II, where Marc Garlasco, former chief of high-value targeting at the Pentagon, visits Iraq to see some of the actual sites he helped plan to hit.
posted by Brandon Blatcher (24 comments total)

The Marc Garlasco bit starts at about 24:45.
posted by Brandon Blatcher at 7:52 PM on November 12, 2006

Direct TAL link.
posted by Armitage Shanks at 7:55 PM on November 12, 2006

Thanks Armitage.

Also note that the Garlasco bit is NOT in Act II, it's in Act I.
posted by Brandon Blatcher at 8:00 PM on November 12, 2006

If this is the episode on the radio a while back, I think it's one of the best pieces of journalism on the war in Iraq that I've heard during the duration.
posted by weston at 8:08 PM on November 12, 2006

I listened to this early in the week, and it is truly spectacular. Hearing the process of how the Lancet report got "discredited" was truly stunning.

(It seems to me like TAL is getting more discussion just about everywhere now that you can link to actual recordings of episodes...)
posted by Matt Oneiros at 8:13 PM on November 12, 2006

Excellent link, thanks.
posted by teece at 8:58 PM on November 12, 2006

nprFilter. heard it. love it. i hope mefi commune is able to find and hear things like this by themselves, but i am glad you've shared it...
posted by localhuman at 8:58 PM on November 12, 2006

localhuman, yhbt.
posted by Space Coyote at 9:05 PM on November 12, 2006

Publishing that study was the most effective advertising that the Lancet has ever done for itself.
posted by smackfu at 9:06 PM on November 12, 2006

Publishing that study was the most effective advertising that the Lancet has ever done for itself.

Is the premise that Lancet only published the study for the attention it would bring, and not for the quality and importance of the research?
posted by Blazecock Pileon at 9:43 PM on November 12, 2006

I certainly didn't say that.
posted by smackfu at 9:58 PM on November 12, 2006

I certainly didn't say that.

Perhaps not, but your implication seems clear. So, what exactly did you mean, if not to smear the motivation of the lancet?
posted by Rumple at 11:11 PM on November 12, 2006

Part of a transcript from yet another radio program:

Another criticism of the method is that your survey suffers from what's called a main street bias, that in selecting households your teams move outwards from the centre of towns where the majority of violence is likely to occur, and so it's therefore an unrepresentative sample.

Les Roberts: Yes, I heard this, it's coming from a couple of physicists at Oxford, and I just simply think it's not true. Every time you do a survey like this, there is the possibility that biases creep in, but this one almost certainly isn't possible. And I say that for a couple of reasons. First of all when we picked a house on a randomly picked street, we almost always ended up two or three blocks away to get to the 40th house. So the one that was picked was not the majority of houses in any given cluster, that's No. 1.

No. 2, we had an equal chance of picking a main street and a back street, and No.3 the vast majority of deaths are violence, the vast majority are adult males, and it seems, we didn't actually collect this information, but when bouncing this main street bias idea off of the doctors who did the work, they said that the vast majority of the deaths happened when people were away from their homes. In markets, standing in queues, walking on the streets going back home. So it just doesn't fly.

You've also come in for criticism from a group called Iraq Body Counts. Now Iraq Body Counts are very much opposed to the war in Iraq, and they're a group of volunteers who keep a record based on public sources of reported deaths in Iraq. Their figure of the current number of deaths is around 50,000; in other words it's less than 10% of your figure. How do you explain that discrepancy?

Les Roberts: So they started out by culling through newspaper reports and trying to record all of those events that appeared in two different media reaped sources. And after a while they managed to get some reporters to intentionally go visit the morgue in Baghdad and other places. But first of all, that sort of passive surveillance process is never complete in a time of war. In Bosnia the best estimate is about 30% to 40% of deaths were captured by hospitals, plus morgues, plus all the governmental reporting systems. But usually in times of war, it's far, far lower. So that's thought No. 1.


Thought No. 2 is if Iraq was the healthiest country in the world, just no violence, no troubles at all, there have to have been by this point, about 500,000 deaths from natural causes, since the US-led invasion, in March of 2003. So if there's been half a million deaths from natural causes what fraction of those have been reported by the Ministry of Health and are captured by hospitals and morgues? And it's not a tenth, it's a tiny fraction. And this actually leads in my mind to the most interesting thing about our study. This is probably of any major scientific study in recent history, whether it's whether Dolly the Sheep was cloned, or whether there was cold fusion, this is the easiest one for the press to either debunk or verify, because if Iraqi body count is correct, if you go out to the graveyards and morgues across Iraq, only one in ten bodies coming in must be from violence, and 90% must be from natural causes. If we're right, the majority of bodies over the last three years that have come in, must have been from violence. And it would be pretty darn easy to go to four or five or six places across Iraq and get a quick pulse on whether or not it's a tenth of all deaths that are from violence, or most of all deaths that are from violence.

The Iraq body count also points to another bit of research done by the United Nations, which surveyed more than 20,000 households, now they too came up with much lower figures for violent deaths than your smaller survey.

Les Roberts: So let's back up, you know virtually all of the criticisms you've just mentioned coming from various groups in and around Oxford, are actually the same group criticising, and this one is by far the most disingenuous because that survey that you cite was done by a great researcher out of Norway, and it wasn't a survey about mortality, it was a survey about living conditions, and they asked a whole host of questions, the average interview I believe took 82 minutes. There was one question about deaths; if someone had died, then there were a couple of follow-on questions. And they found in the first 13 months of occupation, that 20-something-thousand people had died, and they said 'Look, it's so out of whack with what these two Lancet studies are saying.' And that's just not true, they're not very far out of whack. First of all, it was Iraq governmental employees from Saddam's era that was doing the interviewing, and we're pretty sure they didn't get complete reporting of deaths. We're sure of that for two reasons: 1) the overall death rate they came up with was about half the overall death rate we came up with before the invasion, and at the end of this study, the UNDP study site, the lead researcher, Jan Patterson, was sceptical about this mortality data, so he sent the interviewers back to the exact same houses and had them just ask about deaths under five. And lo and behold, when people met these interviewers for the second time when they were a little more comfortable, they reported 50% more deaths on the second interview than they did the first. So the researchers know their death estimate was a gross underestimate. And in the first year of the occupation, the death rate was very low compared to the second year. In fact in our study that came out in 2004, there were twice as many violent deaths in the last six months of the occupation that we studied, as there were in the first 12 months. So the two studies actually only differ by a factor of about 2, and the UNDP study which was not about mortality, knows that it was a gross underestimate of deaths.

Counting the dead in Iraq

See also

To its credit, the BBC website has tried harder than most mainstream media to report the issue honestly. In particular, BBC world affairs correspondent Paul Reynolds - who has frequently engaged with Media Lens readers - responded to complaints by agreeing to invite questions from members of the public and to forward them to the authors of the Lancet report. On October 30, the BBC posted an edited version of answers from Les Roberts... Below, we are publishing Roberts' unedited answers. We have also added Roberts' response to an editorial by Steven Moore in the Wall Street Journal.

MediaLens: Lancet Report Co-Author Responds To Questions

See also Tim Lambert: More Answers From Les Roberts

See also Iraqanalysis.org
posted by y2karl at 11:53 PM on November 12, 2006

I was shocked myself when I saw the figure of 650,000. It seemed huge, much larger than I had imagined possible. It is approximately four times the Iraqi Health Ministry's recent estimate, and twice the figure of 300,000 that is often given as an estimate of the number of people killed by Saddam Hussein during his 23 years of brutal rule...

Are these numbers credible? I looked at reactions to the Lancet study from several groups: American political pundits, scientists with expertise in health and mortality research, and Iraqis (as reflected in the views of Iraqis with English-language weblogs). Many of the political pundits (even those with anti-war views) either rejected the study or questioned its conclusions and methodology. The scientists, however, gave it high marks, and most of the Iraqis thought the number sounded like it was in the right ballpark...

It is rather striking, moreover, that critics of this research have mostly avoided calling for additional, independent studies that could provide a scientific basis for either confirming or refuting its alarming findings.

Center For Media And Democracy: At Long Last, Can We Please Start Counting the Dead ?

So, we have a figure that is approximately four times the Iraqi Health Ministry's recent estimate, and twice the figure of 300,000 that is often given as an estimate of the number of people killed by Saddam Hussein during his 23 years of brutal rule and, as noted before, one tenth of six million--the number commonly given for the total number of Jewish victioms of the holocaust. There has been next to no press coverage of the study, next to no calls for an independent investigation to prove or disprove the study's findings and next to no use of the most appropriate word for violent civilian deaths in such numbers: genocide.

The truth is out there but if it's just too horrible to contemplate, can we handle the truth ? It's a collective reversal of the line from Fox Mulder's wall poster: We Don't Want To Believe.
posted by y2karl at 12:30 AM on November 13, 2006

From y2karl's Deltoid link, quoting Dr. Roberts, perhaps addressing smackfu's issue (whatever that issue was):

Almost every researcher who studies a health problem is opposed to that health problem. For example, few people who study measles empathize with the virus. Thus, given that war is an innately political issue, and that people examining the consequences of war are generally opposed to the war's conception and continuation, it is not surprising that projects like these are viewed as being highly political. That does not mean that the science is any less rigorous than a cluster survey looking at measles deaths. This study was the standard approach for measuring mortality in times of war, it went through a rigorous peer-review process and it probably could have been accepted into any of the journals that cover war and public health.

The Lancet is a rather traditional medical journal with a long history and is not seen as "left-wing" in the public health and medical communities. The types of different reports (medical trials, case reports, editorials) in the Lancet have been included for scores of years. The Lancet also has a long history of reporting about the adverse effects of war, and the world is a more gentle place for it. [emph. added]
posted by Blazecock Pileon at 12:34 AM on November 13, 2006

I really appreciated the fresh look at the Lancet study the episode offers, but I think it'd be a loss if that's all people thought it was about or took away from it. It's much more than that.

Garlasco talks about the process of making targeting decisions he knows will be associated with certain number of human deaths. The military planners know there's a cost associated with any of their targets. Higher-ups -- even the President, if I recall correctly -- have to sign-off on scenarios where projected civilian deaths exceed a certain number. The fact there's a calculus here is both encouraging -- because you know there's limits and a reckoning of costs -- and disturbing that life and death are being worked out almost with the dispassion reserved for producing spreadsheets.

Garlasco does make his own work real by standing on the ground and thinking about it, though. The glimpse into how this affects him isn't as thorough as I'd like, but it's still fascinating.

And then, if I recall correctly, in Act III, they take a look at how our troops have to deal with civilian deaths while trying to do the job of securing places and winning over the populace.

It all makes the civilian death numbers a much more immersive, human, and thoughtful experience.
posted by weston at 1:25 AM on November 13, 2006

The military planners know there's a cost associated with any of their targets. Higher-ups -- even the President, if I recall correctly -- have to sign-off on scenarios where projected civilian deaths exceed a certain number.

It was astonishing! The number is 30, but there is a problem.. Every aspect of the targeting models, save one, is confirmed with on the ground data collected after the fact. Civilian deaths are part of the targeting models, but apparently no one has ever attempted to correlate on the ground civilian casualties with the model projections.
posted by Chuckles at 1:33 AM on November 13, 2006

Chuckles -- That's exactly why Garlasco quit his job as a military targeter and joined Human Rights Watch...
posted by footnote at 6:55 AM on November 13, 2006

On the civilian deaths by Garlasco:

what a flawed method. Less that 30: bombs away! More than thirty: President or DefSec must sign off.

Way too course.

But weston, the second and third parts of this interview were very good, too.
posted by teece at 8:30 AM on November 13, 2006

It was originally broadcast in October of 2005. Obviously still relevant but I'm sure civ casualties are now much higher.
posted by JJ86 at 9:54 AM on November 13, 2006

European, started listening to TAL since they opened their podcast feed three episodes ago and can I just say this: that is the most amazing radioshow I have ever heard. So is this a memorable episode or are they all this good?
posted by thijsk at 12:59 PM on November 13, 2006

Almost all are as good. It is about the best program on air anywhere.
posted by y2karl at 1:24 PM on November 13, 2006

Almost all are as good. It is about the best program on air anywhere.

Hear, hear.

I can't wait to listen to this when I get home.
posted by Sandor Clegane at 4:11 PM on November 13, 2006

Critics of the Lancet study suggesting hundreds of thousands of deaths in Iraq since the 2003 invasion should call for additional, independent studies that could provide a scientific basis for either confirming or refuting its alarming findings.
posted by homunculus at 8:51 PM on November 18, 2006