Iraq Body Count’s Distortions

Iraq Body Count, the organization that has been publishing gross underestimates of the number of civilian dead in Iraq since 2003, is pissed at the Lancet for giving a more accurate figure. IBC has written a lengthy apologia to that effect, which boils down to pointing to another survey that gives a lower figure.

IBC’s apologia talks about the Iraqi Living Conditions Study:

[Link] Discussion of ILCS (Iraq Living Conditions Survey, also known as IMIRA [Iraq Multiple Indicator Rapid Assessment] or the UNDP study, published May, 2005), has been minimal among IBC’s critics and generally falls into two contradictory camps.

The first camp asserts (wrongly – see section 3.6.2) that ILCS perfectly corroborates Lancet, and ends discussion of ILCS there.

The ILCS is a perfectly good study, if you want to learn how many Iraqis own washing machines or what the Iraqi wage gap is (working women make twice as much as working men per hour; however, women are only 16% of the labor force). What it does not give is death rates after the war, except for infant and maternal mortality.

IBC’s defense of the ILCS is based on one small tidbit from the analytical report‘s page 54 (55 in the PDF file), which states,

The number of deaths of civilians and military personnel in Iraq in the aftermath of the 2003 invasion is another set of figures that have raised controversy. The ILCS data indicates 24,000 deaths, with a 95 percent confidence interval from 18,000 to 29,000 deaths.

(…)

The ILCS data has been derived from a question posed to households concerning missing and dead persons during the two years prior to the survey. Although the date was not asked for, it is reasonable to suppose that the vast majority of deaths due to warfare occurred after the beginning of 2003.

The ballpark figure of 24,000 is a good reason to doubt the relevance of the data to Lancet. The Lancet study estimates a total mortality rate, which in principle is cause-blind. When asked about war deaths, people may not respond affirmatively if the death was not obviously related to the war.

Obviously, if your mother was killed in Shock and Awe, you’ll almost certainly consider her a war dead. But what if she was killed by American troops who were frustrated with their not finding any real insurgent? What if she died in an insurgent bombing? The phrasing of the question is skewed toward war deaths as opposed to occupation deaths. At most, it refutes Dana’s assertion that the Lancet numbers are too high because by a certain metric they’re higher than a certain WW2 death toll.

The ILCS doesn’t have any figure for the total death rate, so it’s really incomparable with the Lancet study. But even if it were, IBC’s claim that the ILCS must take precedence betrays ignorance of statistical testing. The meaning of “the 95% confidence interval of the 650,000 figure is 400,000-900,000″ is “If the real figure is between 400,000 and 900,000, then the Lancet study’s methodology gives a 95% confidence interval that includes 650,000.”

In other words, the Lancet study may have a large error margin, but it also has high enough a figure that it doesn’t matter. From the above formulation, if some group releases a study that says 400,000 Iraqis died, then the Lancet’s figure is within its margin of error, so we can’t conclude the group is wrong. But if a group claims that 40,000 Iraqis died, then the Lancet’s figure is well outside its margin, so the Lancet contradicts it.

When we have two contradictory studies, we can’t ever assume that the one with the larger sample size is wrong. We can only assume that if we have some discrepancy that lies within the margin of error. If the discrepancy is this big, we need to investigate the methodologies and see who’s doing a mistake; it’s possible neither side is, but the probability of that is vanishingly small. In this case, we have a study of Iraqi death rates that uses a standard epidemiological methodology, versus a compilation of media reports that not only neglects deaths not reported to the authorities but also neglects deaths not mentioned in the media.

About these ads

11 Responses to Iraq Body Count’s Distortions

  1. llewelly says:

    In some ways, it seems the function of IBC verges on that of an astroturf group.
    Very sad, and I’m sure not at all what the IBC folks intended. But they’re captive to the Myth of Personal Rightness, it seems …
    What, no preview!?!?

  2. Alon Levy says:

    I don’t think it’s astroturf. The people at IBC are anti-war for humanitarian reasons. Like me, they think 40,000 civilian casualties are enough to invalidate the case against the war. But they’re attached to their own numbers, and since they have no vested interest in inflating the numbers – to them, 40,000 is damning enough – they end up having a vested interest in downplaying them in order to appear more moderate.

    No, no preview… WordPress is annoying like that. Sorry.

  3. joeo says:

    If x is right, I am wasting my time.
    I am not wasting my time so
    x is wrong.

  4. [...] So then I was thinking, must I read this because they come up with an equally rigorous study that comes to different conclusions to the Lancet study? And for a while I thought maybe they had, in that they say in their view, there “is considerable cause for scepticism regarding the estimates in the latest study, not least because of a very different conclusion reached by another random household survey, the ILCS, using a comparable method but a considerably better-distributed and much larger sample.” Sounds good until you realise that the ILCS study isn’t quite what they say: The ILCS is a perfectly good study, if you want to learn how many Iraqis own washing machines or what the Iraqi wage gap is (working women make twice as much as working men per hour; however, women are only 16% of the labor force). What it does not give is death rates after the war, except for infant and maternal mortality. [...]

  5. Anatoly says:

    You are quoting the wrong report by IBC, not the one that addresses the recent Lancet study. And you also, it seems, can’t keep track of which report you’re talking about in the latter half of your post, IBC or ILCS. I posted the details in Tim Lambert’s post which links here.

  6. Alon Levy says:

    I know it’s not that one. The one that addresses the recent Lancet study is just a compilation of variations on the argument from incredulity. The people at IBC can’t believe that 650,000 people died; hence, 650,000 people didn’t die.

    But in a comment on Appletree, Fritz pulled that older apologia as an argument against the recent Lancet study. Frankly that’s much stronger an argument than the “No, we don’t believe it” argument, even though it’s still wrong.

  7. Nod. At my blog, I dug into the IBC response, and it all comes down to the same thing. “We surely would have known!”

    That all assumes that a nation on the brink of civil war has nothing better to do than gather death certificates and report on the wounded. It also assumes that Iraqis can all go to the hospital for treatment (and all treatment is getting logged and tracked).

    It asks why there aren’t more wounded… but if Iraqis are afraid to go to the hospital, then more of them will die from their injuries; more deaths, and fewer injuries, and suddenly the lack of injured people doesn’t look as compelling.

  8. Trevor says:

    Ho hum

    If the lancets published figures (655,000) are right than in 3 years we have seen about as many deaths as was in inflicted on the British Army in the Western Front in WW1 (700,000) over 4 years.

    On top of which there would be 3 times that many wounded.

    All inflicted without turning a hair on the heads of the assembled world press. The cavalier way you dismiss the non issue and the non reporting of the non issue of death certificates is breathtaking. The main way the Lancet report justified the accuracy of its figures was the existence of death certificates. If the death certificates exist then there is no need for the interviews. If the interviews do not throw up a lack of death certificated the sampling is grossly in error.

    You go believe that fantasy in your dreamworld if you want.

    Just where are all these dead bodies. The British press are in Basra and district – have they seen the graves or the funerals of the bodies in question and if Basra in the South and the Kurdish North are relatively benign then the situation must be correspondingly worse in the remaining areas – and all completely missed by Aljazeera.

  9. Greg says:

    If anyone is actually interested in the truth, try to eliminate bias when reading the methodologies of each survey. Instead of grasping at the number which best fits your current idealism: “pro-war”/”anti-war”, try using rational thought which demands you use the facts to shape your ideals rather than vice versa.

    Objectively determine which survey is most accurate by reading each defense and coming to your own conclusion as if you don’t care who is right (in order to eliminate bias). As an excersise, just replace “civilian casualties” in your head with something else like “apples”. Not because deaths don’t matter, but because not allowing your emotion to cloud rational thought does. At least if accuracy is what is important to you….

  10. Tyler DiPietro says:

    Yeah Alon, seriously dude. Where did you ever get the idea that it was plausible for a four year long civil war to kill about 600,000 people. Certainly not here. The fact that standard epidemiological techniques were used to get the Lancet number means that we must toss the entire field out as a sacrifice for Bush’s war. In order to get a more accurate number we can just have Bush get into contact with God to find out how many Iraqis ended up in hell for following a false religion.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: