Ashley,

 

Check your math.  A CRM of 5.37 per 1000 equals approximately 140K per year when I calculate it. 

Shek

 

Oh ooopps forgot the extra 0.  139,620 dead per year is correct.  However I still suspect the death rate for 2006 is underreported.  In July 2000 Iraq's death rate was 6.4 per 1000, 6.21 per 1000 in 2001, etc.under Saddam, yes, but internally stable as well.

 

I have also seen UN estimates that Iraq during the 1980s (full blown war and genocide) had a mortality rate of 8.1 per 1000.  The only questions on the study are is the baseline underreported at 5.5 per 1000 (we could say it is 7.5/1000 to keep it in line with previous numbers, and that would reduce the number of  excess deaths by several hundred thousand) , and is the sample biased.  It is doubtful that Iraqis who had their relatives killed by Saddam would under report those deaths in a confidential study.

 

There are currently 100 deaths per day in Baghdad, also there are reports that there are at least one violent death every hour in Basra, and at least that many (24) in  the rest of Iraq.   So at least 55,000 Iraqis will die in political violence per year or close to 200,000 Iraqis dead for the first four years of the war, and those are the numbers we know of.  Not every Iraqi shot in the back and dumped near his family's house is going to be reported in the media or in the morgue.  The Iraqis certainly cannot trust the police to help them "solve" the murders.

It is just SO unfair to be using a left-wing website to question the veracity of the Lancet study.

 

Here's another link to them as well:  link

Reality checks: some responses to the latest Lancet estimates

Hamit Dardagan, John Sloboda, and Josh Dougherty

Summary

A new study has been released by the Lancet medical journal
estimating over 650,000 excess deaths in Iraq. The Iraqi mortality
estimates published in the Lancet in October 2006 imply, among
other things, that:

1. On average, a thousand Iraqis have been violently killed every
   single day in the first half of 2006, with less than a tenth of
   them being noticed by any public surveillance mechanisms;


2. Some 800,000 or more Iraqis suffered blast wounds and other
   serious conflict-related injuries in the past two years, but less
   than a tenth of them received any kind of hospital treatment;


3. Over 7% of the entire adult male population of Iraq has
   already been killed in violence, with no less than 10% in the
   worst affected areas covering most of central Iraq;


4. Half a million death certificates were received by families
   which were never officially recorded as having been issued;


5. The Coalition has killed far more Iraqis in the last year
   than in earlier years containing the initial massive "Shock and
   Awe" invasion and the major assaults on Falluja.

If these assertions are true, they further imply:

• incompetence and/or fraud on a truly massive scale by Iraqi
  officials in hospitals and ministries, on a local, regional and
  national level, perfectly coordinated from the moment the
  occupation began;


• bizarre and self-destructive behaviour on the part of all but
  a small minority of 800,000 injured, mostly non-combatant,
  Iraqis;


• the utter failure of local or external agencies to notice and
  respond to a decimation of the adult male population in key
  urban areas;


• an abject failure of the media, Iraqi as well as
  international, to observe that Coalition-caused events of the
  scale they reported during the three-week invasion in 2003 have
  been occurring every month for over a year.

In the light of such extreme and improbable implications, a

rational alternative conclusion to be considered is that the
authors have drawn conclusions from unrepresentative data. In
addition, totals of the magnitude generated by this study are
unnecessary to brand the invasion and occupation of Iraq a human
and strategic tragedy.

That's a very interesting article, wbull. 

I definively recomend anyone interested in the subject read that, along with
- The survey itself:    i.a.cnn.net/cnn/2006/images/10/11/human.cost.of.war.pdf  (cut & paste into browser)
- It's appendix and charts:   i.a.cnn.net/cnn/2006/images/10/11/appendices.human.cost.of.war.pdf  , and
- The relevant CIA figures (above, posted by Shek)

Coming back to the article -
David Frum is unabashedly partisan, in addition to being a profesional political hack. Crafting partisan arguments is what he does best - that actually is his line of work. You'd expect that if anyone out there would be able to provide a compelling summary of the arguments againt the Lancet study, that would be Mr Frum.

And of course the authors of the Lancet report are NOTpartisan in any why, altruistically publishing their "findings", so they are immediately credible, despite the fact that we have shown on these pages their motives for releasing their report? Cant have it both ways -  they are either credible on the merits, or you make a partisan ad hominem attack on David Frum for being partsisan, in a feeble effort to discredit his analysis before it even gets started. The bottom line is that it is up to the Lancet people to defend their methodology and findings, which to this point they have NOT done, other than to publish the report.

He does show very good technique - for intance, he refers to the Survey as "the Bloomberg study" throughout his article.  That's inaccurate; at least four institutions (the Jhons Hopkins School of Medicine, the U of Bagdhad medical school, MIT's Center for International Studies, and the Lancet's scientific review board) were directly involved in the production and vetting of this survey. 

Not only that, the term "the Bloomberg study" is clearly aimed at undermining the credibility of the research - who the heck is this "Bloomberg team" anyway?.  Though the reader may be willing to grant a certain amount of credibility to some of the world's best scientific institutions (like Jhons Hopkins, the Lancet or MIT), it's much easier to poo-poo "the Bloomberg team"

However, on the substantive facts, Mr Frum doesnt do nearly as well:

Point 1 - "Most Iraqui experts" estimate the casualty rate as 30-60,000 dead, while"the Bloomberg team" estimates 655,000.  Hence "the Bloomberg team" is not credible.

"Phooey!" to that.

Nice concise English, btw. About as much as your "experts" on the Lancet report.

For one, the Lancet study involved far more "Iraqi experts" than almost all the unnamed competing studies cite by Mr. Frum. That included 10 Iraqi MDs doing months of field reseach in every single province of Iraq, in addition to the Medical School of the University of Bagdhad activally working alongside both Jhons Hopkins and MIT.
 
For another, the studies reporting 30-60,000 dead are based on "secondary source" aggregation methodologies (adding up deaths reported by press reports and morgue records).  That's an *absurd* approach

Please explain that to us poor statisticians - the obviousness eludes us, since it is attempting to define the universe, which in many ways is a sounder approach than extropolating from a VERY small smaple size.

 

to apply to measuring death rates in active armed conflicts, for obvious reasons.  Not only that, secondary-source studies are well know

By WHOM? Your sayso?-  PHOOEY TO YOU.  Please cite some authorities on this.

 

B to report under 20% of the caysualties caused by large-scale armed conflicts (the only known case of an armed conflict where secondary source aggregation studies have identified over 20% of the actual casualties was in Bosnia)

Point 2 - "The Bloomberg team" "hired Iraqui interviewers" who went out to 1850 houses, reported only 547 excess deaths.  It's unreasonable to extrapolate 655,000 excess deaths from such sketchy &scanty data.

Double "phooey", with a cherry on top.
For one, those sketchy "hired Iraqis" used by "the Bloomberg team" were 2 teams of 5 MDs each provided by the Medical School of the University of Bagdhad. Along with support personel, they traveled to every single province in Iraq and carried out an extensive randomized population-adjusted survey, including 1850 households and over 12,000 people.  Those are not sketchy "hired Iraqis"; it's *by far* the best survey team ever deployed to measure mortality in post-war Iraq.

And you know this as a fact?  No ulterior motives, no breakdown in record keeping, it boggles the mind. Please show this  - or again, phooey back to you.

For another, finding 629 deaths (of which 547 are 'excess' deaths) in a random sample of 1850 households is an extremely relevant   number. Mr Frum's argument seems to be a Luddite "how can ya go from 547 to 655,000?" and the answer is absolutely; you can. 

More ad hominem attacks - it weakens your already weak argument.

 Coming from somebody as familiar with political polling as Mr Frum, the argument is disingenous in the extreme.

Triple hooey-   as is yours argument.

Point 3 "The Bloomberg team" has "a record of doing their work improperly", because of the 2004 survey. 

Well, well.  Triple "phooey", with chocolate swirls & candy sprinkles on it.
The top-line conclusion of the 2004 survey was that there were 100,000 excess deaths after one full year of armed conflict across *all* of Iraq (a nation of 22,000,000).  It absolutely was not that there were 200,000 excess deaths in one small city -that's quite simply a fabrication.
Where he gets this is from is the fact that within that nation-wide survey, one data set (that refering to population living near Fallujah) was a statistical 'outlier', that is, not coherent with the rest of the survey's findings.  Because of that, during the statistical analysis portion of the survey, the Jhons Hopkins medical school team decided to exclude that data set.  Not only is that good statistics (the exclusion of outliers is a bog-standard aspect of statistical analysis), it's actually a conservative approach that tends to minimize (not increase) measured casualties. 
Furthermore, the Jhons Hopkins team published not only their top line results but also the methodology followed and even the data sets that were *excluded* from the final results.  Far from being "a record of doing their work improperly", that's the hallmark of a carefully done and thoroughly documented scientific research.

Which is why the first Lancet report was widely and thoroughly debunked and discredited when it came out - by both the left and the right.

Etc., etc. - We could easily go on, point by point, through each one of Mr.s Frum's arguments but I think the point is well established. 

NO-  only YOU think the point is well established - i dont think youve convinced too many people here.

 Mr Frum is higly experienced political hack with an ax to grind,

And, of course,  the Lancet folks are not - such altruistic specimens. And Brutus is an honourable man.

but once you look at his arguments in detail, he fails to substantively undermine the credibility of the research. 

No - only you think that.

The primary reason for that failure is not lack of skill on the part of Mr. Frum - indeed he is very, very good, particularly if you dont pay much attention to detail.  The 'problem' here is that this survey is, far and away, the best research ever carried out into the subject

Something so blatantly and obviously stupid could only be embraced by an intellectual. Sorry George. It might make for great statistics, but sure doesnt pass the sniff test. I could design the best study in the world -( I did market research analysis and designed and analyzed surveys for a living for 5 years), but if you go in with preconceived notions - and there is NOTHING to show that they WERENT unbiased, by their own admission - the whole premise of the survey is suspect. Again, i would want to know more about the actual collection techniques, where and all the "death certificates" were obtained, were these people actually randomly selected or  were the interveiwers "steered"  to these people by the local imam or block leader.  So many questions on technique,  so little time,  but more than enough to cast doubt on the process itself, let alone the conclusions. I'll wait til the final papers come out, and they are held to public and other peer review from NEUTRAL parties.

 - bar none and in all fundamental aspects (survey design, fieldwork execution, statistical analysis and publication standards). 

Not only that, the survey's results (2.5% of the population killed in 3+ years of armed conflict) are enminently reasonable and consistent with what we've seen in previous armed conflicts,

So you say - repeatedly - please cite those sources to back up your assertion.

 while the alternative proposed by Mr. Frum (0.12-0.25% of the population killed in 3+ years of armed conflict) is frankly unbelievable, and would require a fundamental departure from the history of warfare as we know

as YOU Know it.

 it to be even roughly plausible.

Heart,

eu4ea

Have a nice day

swhitebull

 

 

Re: the Lancet study, Shek's recent points were why they would spend time in their report comparing their findings to what we know about mortality in other wars (Congo, East Timor, Vietnam, etc.) and why they only used 14 months of pre-invasion data as their benchmark for the expected death rate.

As I see it, the answer to the first one is quite straightforward; it was an external 'sanity check' to see if their findings were reasonable.  The question there was "are the results we are obtained consistent with what we've seen in previous conflicts?", and the aswer to that was a resounding "yes" - 2.5% of the population dead over 3+ years of armed conflict seems to be highly consistent with what we have seen elsewhere.  If anything, it's unusualy low.

There are two issues here.

First, they attempt to benchmark their final numbers, but they hardly spend time developing this. For example, they cite Vietnam and 3 million casualties. Is that over 25 years? 15 years? 8 years? What about the causality of these deaths? Agent orange? Air strikes? Indiscriminate carpet bombing? The blockade of North Vietnam? Besides making zero effort to give details on Vietnam, they make no effort to motivate that comparing Vietnam and Iraq is a valid comparison.

For East Timor, they cite a number provided from CNN. Do you find this very scholarly? Do you expect researchers well versed in statistics to rely on a citation from CNN? They criticize the IBC method, and yet they engage in using a news source. Coincidentally, their number is the highest number out there, 200K, published by Amnesty International. While I didn’t do an exhaustive search, I couldn’t find any scientific methodology behind the 200,000K number that Amnesty International cites. However, why do they not cite the following? 

link

This study cites 103K as the number of excess deaths over the 25 year period. This would equate of half of the excess death rate claimed by Iraq.  Once again, they provide no details and make no attempt at motivating how East Timor is a good benchmark for Iraq. 

Lastly, they cite the Congo as a benchmark, once again without giving many details that could then be used to motivate why it is a good benchmark for Iraq. As an aside, when you read the study they cite, you find that the methodology used for Congo is different. Furthermore, the study they cite doesn’t even talk about the methodology in computing the wartime deaths. Instead, the study they cite looks at the peacetime (post-war) death rates and talks about how there had been 500K of deaths in the 16 months following the peace, with 0.1% of the deaths resulting from violence. The bulk of the deaths resulted from HIV/AIDS, lack of vaccinations, and malaria. Those are certainly problems in Iraq, not! Either their scholarly research capabilities are not worthy of the doctorates they have, or else they purpose chose black box figures. 

The second issue is that they don’t even touch benchmarking for their pre-war numbers (other than citing their original study, which is slightly incestuous). Given their attempt to benchmark their overall results, don’t you find it disingenuous that there is no benchmarking discussion over their pre-war results, for which there are benchmarks to compare against?

Bottomline, their benchmarking efforts are poor and not sufficient. 

The implication (which they did not state) is that numbers previously presented (such as Bush's claim last December that around 30,000 Iraqis had died) are extremely unusual.  0.125% of the population dead over 3 years of fighting is on the face of it, absurd.  It would be the first time in history that we see anything like that.
I believe that you are making an apples and oranges comparison here on a couple of fronts.   

link

link

First, you are trying to make comparisons across different time periods. Not central to the argument, but sloppy and fallacious all the same.

Next, President Bush hasn’t argued that the “excess” death figure in Iraq is 30,000. When he spoke about the 30K figure last year, he spoke of Iraqi “citizens.” Does this include those who have died from non-violent causes? Does it include Iraqi insurgents, who I’m pretty sure he wouldn’t include as “citizens.” Because he didn’t define his metric, you cannot make direct comparisons (although you can make some plausible assumptions and then compare some components of Roberts et al once you’ve adjusted the figures in one time direction or the other). 

Regarding the question of why they only used 14 months worth of pre-war data as their benchmark, I dont have a good answer for that. I imagine they could have used more, dont know what their reasons were. The related question, however, is how much of a difference this would have made. 

To determine if limiting the pre-war time period would have a bias, you’d have to ask yourself if anything in pre-war Iraq would have been different in 2003 than it was prior to that. I’d argue that the answer is yes. First, you have a dictator that understood that the war clouds were gathering and began to negotiate truces with many tribes so that he’d have a more united front against any potential invasion. So, you have fewer regime related deaths than if you were to have opened up the time period. Next, you have to look what effect the release of tens of thousands of prisoners in October 2002 link had, to include nearly 10K violent crimes prisoners. If you think through this, I believe there is a bias there that would overestimate post-war deaths (i.e. while there would be some extra crimes in the pre-war figure, these didn’t factor in during the entire pre-war period; conversely, these criminals would be reflected in the entire post-war period). Furthermore, despite your claims of causality, you cannot argue that the deaths at the hands of these criminals is a causal relationship with the invasion.

Next, if the authors were worried about recall bias by opening up the time period, they could have just broken the pre-war period into several phases just as they did the post-war period. If they found a large # of unsubstantiated deaths, then they could make a compelling argument that to include the oldest phase would bias they figures and then drop them.

Someone, I think it was EW3 went through and calculated it using a 5 year average of CIA figures - I believe the figure he came back with was 6.1 deaths/thousand pop. (it's in the missing threads).  The benchmark the Lancet used was 5.5 deaths/thousand.  While significant, the 0.6 difference is not central; the average post-war mortality measured by the Lancet was 13.2 dead/thousand.  Hence even if we apply the more stringent benchmark, the difference would account for less than 1/20th of the measured total.

Once again, you are trying to have it both ways. You have yet to demonstrate some methodological change that means that we should accept the pre-war numbers but reject the post-war numbers. Can’t have it both ways! Accept them all, reject them all, or demonstrate some change that would result in accepting the pre-war while rejecting the post-war – what’s it going to be?

Which comes back to the central point, which is that even allowing for the issues we've located so far (the clustering, the empty-house problem, the two excluded datasets, and the lenght-of-benchmark) the Lancet study is still by a *long* shot the best study of post-war mortality I have seen so far. 

Most of the rest have glaring base errors - for one, most are simple secondary source aggregates, which is simply not a relevant method, by any stretch. Not only that, their results  cluster around 0.1 to 0.4% of the population dead in  3+ years of armed conflict, while the CIAs numbers actually claim that mortality has steadily gone down over the course of the war. To put it mildly, those results are radically out of synch with what we've seen in other conflicts.  Further, the documentation about their study design, field work and statistical analysis is either non-existent or a dramatically lower quality than the Lancet's.

The cluster sample design doesn’t necessarily affect the accuracy of the study – it does make it very imprecise, however. I am not raising this an issues. The empty-house problem doesn’t necessarily present a problem. While it could present a survivor bias problem (which would result in underestimation of the mean), it could also reflect a bias towards overestimation of the mean (e.g. a family that’s taken a sectarian holiday to avoid violence by going to a friends or relatives house elsewhere – this would be a double whammy – as it would result in overweighting the cluster as well as missing a family where you had fewer deaths than the mean), or it could be a wash. A problem, but I think one that at this point is hard to find a distinct bias in a particular direction. As far as the benchmarking, I have a significant issue with the Roberts et al study that completely undercuts their numbers. 

I would agree that this study has the best post-war results of mortality in Iraq mainly because they look at total mortality and not at more specific subgroups that other sources monitor. However, as the only post-war study of the crude mortality rate, it’s not hard to get the label “best.” Once you benchmark their pre-war rates, you could reduce the excess death rates by over 1/3, and now that you are playing on the margin, I would find it plausible that you could move the rate even further down such that you would fail to reject the null hypothesis at the 5% level of significance. Maybe not, but it’s possible.

Next, I will take issue with the documentation. IBC’s documentation is vastly superior to Roberts et al. You can search their database online and are free to draw the conclusions from the primary source data. With Roberts et al., all we get is an 8 page study with some frivolous benchmark and no underlying data. No ability to replicate their results. No ability to determine if the field researchers added a death post-war every 5^th house, or whatever. 

Taking all that, at this point it looks like on one side we have a well-designed, well documented study published in a top tier scientific journal that produced results that are very much in line with what we have seen previously. 

Once again, why are Vietnam, DRC, and East Timor good benchmarks? The study made no compelling argument (in fact, I’d argue that they made no argument and instead threw up conflicts with high numbers for propaganda effects relative to their own research). So, the “in line with what we have seen previously” has not been established or proven.  What we do know is that their pre-war numbers are NOT “in line with what we have seen previously.”

“pdf.wri.org/wr98_hh2.pdf”

Next, it is not well documented, whether you want to point to the sloppy and/or misleading references I already pointed out or point out the fact that the data is not available to attempt to replicate the results and to see what measurement biases may or may not exist. I’ve already beat this horse.

Additionally, in the original thread, you called the methodology used by Roberts et al as the “gold” standard in survey methodology. This is wrong. The “platinum” standard would be to survey the entire population. The “gold” standard would be identifying strata and then conducting random sampling with the strata. The “silver” standard would be just a simple random sample. Finally, I’d classify the methodology used in the Lancet study as the “bronze” standard – cluster sampling will yield imprecise results, which is exactly what you see here. 

Next, how random is this? 

“The interview team were given the responsibility and authority to change to an alternate location if they perceived the level of insecurity or risk to be unacceptable.”

This is a violation of random sampling when you don’t sample the random site selected. Not quite the impeccable execution of random sampling that you claimed way back. Now, it is arguable as to what bias this introduces. Does it mean that the mortality rate was underestimated, because you reason that if it’s too insecure to sample, it must have had a lot of deaths? Does it mean that the mortality rate was overestimated, because the interview team were Sunni and didn’t want to move through a police checkpoint into the “randomly” selected neighborhood that happened to be Shia for fear of their own safety? Or was the team Shia and didn’t want to move through an Iraqi Army checkpoint into an all Sunni neighborhood? I could continue on with the permutations of the implications of this, but we can’t figure out the underlying truth because the information is not revealed to us in the level of detail necessary to analyze this properly. Why not?

On the other we have a collection of methodologically unsound, poorly documented studies published across a motley collection of websites and newspapers whose main conclusion is that what we are seeing in Iraq is a level of mortality that's radically lower than any other armed conflict we've ever seen, by a factor of 10-20.

Once again, I’ve already covered your errors here – you are comparing apples and oranges and instead of addressing the weaknesses of the Roberts et al study, you are trying to distract the topic by diverting to other studies that are looking at different measures.

However, I’d state (again) that the IBC is better documented by orders or magnitude. You can search their database and see EXACTLY how they come up with their numbers. Nor do they attempt to hide the limitations of their methods.  Their process is completely objective and transparent. Their methodology is very sound with regards to the results that they advertise.  They don’t claim to have the full answer. Not so with Roberts et al. Their data lies in a black box that yields only results. No full disclosure, no full transparency.

As far as the apples and oranges comment, it is clear that IBC will not report any deaths as a result of health issues that Roberts et al may capture. Also, as I’ve motivated several times now, your benchmarking (e.g. the 10-20 times lower) is misled by the “benchmarks” that Roberts et al purports.  Make an argument on why Vietnam, DRC, and/or East Timor are good benchmarks, and I’ll accept them as a valid comparison.  

Given that, if we had an office pool going, I'm very clear about where I would put my money.

Yes, we all know. Irrelevant, however. You’ve claimed that it’s a great study, and yet beyond cherry picking benchmarks and appealing to authority, you have to discuss details of the study in-depth.