Where he gets this is from is the fact that within that nation-wide survey, one data set (that refering to population living near Fallujah) was a statistical 'outlier', that is, not coherent with the rest of the survey's findings.  Because of that, during the statistical analysis portion of the survey, the Jhons Hopkins medical school team decided to exclude that data set.  Not only is that good statistics (the exclusion of outliers is a bog-standard aspect of statistical analysis), it's actually a conservative approach that tends to minimize (not increase) measured casualties. 
Furthermore, the Jhons Hopkins team published not only their top line results but also the methodology followed and even the data sets that were *excluded* from the final results.  Far from being "a record of doing their work improperly", that's the hallmark of a carefully done and thoroughly documented scientific research.

To claim that excluding the Fallujah data was done in a professional and scientific manner is laughable.  To have included it would have totally tanked their findings, as the Fallujah outlier was so large as to be completely indefensible.  Yet, you could see how much it pained them to exclude it, as they continued to cite results using the Fallujah cluster despite the fact that it wasn't plausible (keep in mind that the study was completed prior to the November 2004 Operation "Al Fajr").  The Fallujah results are discussed in 5 of the 7 pages of discussion from the study article.  If the results were such an outlier, why discuss them in such depth across nearly the entire paper while making statements that maybe it might not be such an outlier (as if other neighborhoods would yield an extrapolation of 56K KIA!).

 

Here's what I wrote last year on this:

 

 Fallujah. “In Falluja, the team noted that vast areas of the city had been devastated to an equal or worse degree than the area they had randomly chosen to survey.”

 Here are some calculations that make the above statement appear very biased. Now, since I don’t have access to their data, I use the average persons per cluster calculation below to extrapolate the findings’ predicted number of deaths in Fallujah. This means that the calculations will most likely be slightly off, but that doesn’t detract from the solid conclusion that Fallujah is an outlier and that the study in the same breath calls it an outlier while implying that it’s the norm as well.

 7868 people surveyed / 33 clusters  = 238 people per cluster

 238 people per cluster / 256K people living in Fallujah = sample is 1/1075^th of the population

 1075 x 52 violent deaths = 55,900 Fallujans dead

 If there’s that many dead, what is the number wounded? 

 So, if you look at the calculations above coupled with their qualitative statement, the conclusion that is drawn from the study is that surveying a different neighborhood means that they still would have arrived at sample statistics resulting in at least 55,900 Fallujans dead. WTF?

Where he gets this is from is the fact that within that nation-wide survey, one data set (that refering to population living near Fallujah) was a statistical 'outlier', that is, not coherent with the rest of the survey's findings.  Because of that, during the statistical analysis portion of the survey, the Jhons Hopkins medical school team decided to exclude that data set.  Not only is that good statistics (the exclusion of outliers is a bog-standard aspect of statistical analysis), it's actually a conservative approach that tends to minimize (not increase) measured casualties. 
Furthermore, the Jhons Hopkins team published not only their top line results but also the methodology followed and even the data sets that were *excluded* from the final results.  Far from being "a record of doing their work improperly", that's the hallmark of a carefully done and thoroughly documented scientific research.

To claim that excluding the Fallujah data was done in a professional and scientific manner is laughable.  To have included it would have totally tanked their findings, as the Fallujah outlier was so large as to be completely indefensible.  Yet, you could see how much it pained them to exclude it, as they continued to cite results using the Fallujah cluster despite the fact that it wasn't plausible (keep in mind that the study was completed prior to the November 2004 Operation "Al Fajr").  The Fallujah results are discussed in 5 of the 7 pages of discussion from the study article.  If the results were such an outlier, why discuss them in such depth across nearly the entire paper while making statements that maybe it might not be such an outlier (as if other neighborhoods would yield an extrapolation of 56K KIA!).

 

Here's what I wrote last year on this:

 

 Fallujah. “In Falluja, the team noted that vast areas of the city had been devastated to an equal or worse degree than the area they had randomly chosen to survey.”

 If there’s that many dead, what is the number wounded? 

 So, if you look at the calculations above coupled with their qualitative statement, the conclusion that is drawn from the study is that surveying a different neighborhood means that they still would have arrived at sample statistics resulting in at least 55,900 Fallujans dead. WTF?

swhitebull

Swhitebull,

 

They don't state as much (that other neighborhoods would have yielded the exact same result), but the strong implication is there IMO. 

 

To me, it is simple, if the outlier status is questionable, then it is fair to discuss the with and without.  When the outlier status is firmly locked, such as a neighborhood that equates to 56K killed when the most that had ever been operating in the area had been a brigade plus, to drag on the discussion is outlandish.  Talk about how the data points from the cluster don't extrapolate into a result that checks with common sense, and then leave it at that.  Don't belabor the point by continuing to discuss with/without results as if that will somehow make 56K killed begin to make sense and seem plausible. 

"Coming back to a more 'serious' level.... Given all that, I find it a little hard to comprehend how and why you'd think that their critique of the Lancet's statistical methodology would be any more credible than, say, Tickle-me Elmo (arf!.. arf-arf!... ooops, sorry)."
If their observation is correct, then why should it be disqualified.

========
Right you are.  I just find it funny that a group whose notorious characteristic is that they use comically poor statistical methodologies would be cited as source of insightfull questions regarding statistics.  However, you are right; address the question, not the questioner.
==============

"Regarding you quotes - the one you posted was not from the Lancet, but from Iraqbodycount.org's own text in their "rebutal".  Which is great, but not particularly informative of what the Lancet published."

You see, I quoted what IBC said and provided reference. Then you I said where they got the number (in The Lancet). That was obviously not specific enough for you.

"Regarding your reference to the Lancet's study, well, hum...it's not in page 12 of the Lancet Survey, as you claim. Correct me if I'm wrong, but I think we're both reading the same document, and Page 12 contains the "Limitations" chapter and half of the "Conclusions" chapter.  Neither one of those even mentions the words "male" or "female", let alone make the claims that you state they do."

Take a look at the graph on p.12. Here is the quote, also p.12:

"In the next graph is shown the deaths from violent causes by age and sex. As can be seen, violent deaths account for most of the deaths, and violent deaths are almost entirely in males. Among the males, there were no practical survey methods to determine which of the deaths were among active combatants. It is interesting to note that the largest single age group of female deaths was among the under age 15 years."

"It's possible that it's elsewhere on the Lancet's survey - if you find it please let me know."

Done.


"In the meantime, I dont follow how you've concluded that a population that has suffered 2.5% mortality could *possibly* have a 7% mortality amongt males, even if every single casualty was male (which they were not).  Unless that happened *and* the population was overwhlmingly female (which it isnt)."

The other way round. Adults constitute less than 66% of pop. Of this group 50% are males. So this group comprisese less than 33% of entire pop. However, the survey found almost all violent deaths were suffered in this group. So

33%*7.5% = 2.5% = Lancet claim of nat'l average.

The actual numbers are

"Of the 287 violent post-invasion deaths recorded by the Lancet authors where the age and sex was known, 235 (82%) were adult males between 15 and 59 years old. Extrapolating to the population as a whole would mean that around 470,000 men in this age group have been killed violently, i.e. one in 15 (7%) of adult males aged 15 to 59"

link target="_blank">link even without the 60+ segment (as I read it).

So the numbers are consistent either way.

7.5% amongst adult males, or more than 7% according to IBC.


"The only way I can see around that is if you only count males in demographic groups where the impact of the war is particularly severe (of a particular age group, of a particular ethnic group, living in a particular location, etc..)  In which case, that claim has no meaning.  If you cherry-pick like that *of course* you're goign to find populations that have a much higher death rate.  Gee whiz, no wonder.

And in no way is that a disqualification of the Lancet's results - of course armed conflict afects certain demographic groups more heavily. For instance: military age males in places where the fighting is hardest tend to get the short end of the stick more often than 50-year olds living in peacefull provinces. That is, quite simply, the nature of warfare - dunno about the guitarists and retired librarians at Iraqbodycount.org, but I'd be truly amazed if anyone here would find *that* to be a surprising result...

Dont want to put words in your mouth, so I'd love to learn more about what this surprising result is and how you came to it."

Of course you wouldn't. I was just responding to this:

"- The Lancet survey does *not* indicate that 7% of the male population of Iraq has died over 3+ years of armed conflict.   The good folk at Iraqbodycount just made that one up.
Let's "reality-check" the math: since the Lancet survey found that 2.5% of the overall population of Iraq died over the course of the war, even if every single casualty was male that still does not account for 7% of the males. Oops!"

Anyway, it makes it possible to discuss if 1:10 to 1:15 are realistic numbers for mortality.

Cheers

And now it is 3pm, erk!

Regardin the sample size.  On this I think you're plain wrong; a randomized sample of 12,000 people in 1850 households most certainly *is* large enough to provide adecuate sample data for a population of 22 million.  I'm amazed you think otherwise if you have worked in a professional capacity in statistics - but let me turn that question around to you; what woul *you* think is an adecuate sample size for measuring mortality rates in a country of 22 million or so?

eu4ea,

 

I have never heard a sample described as randomized before.  This was not a random sample.  This was a multistage cluster sample that used random sampling within the clusters.  There is a difference, and that difference is huge. 

 

However, your underlying point that the sample size was large enough to produce results is fine; however, because of its cluster design, the poor precision of the results make it much more sensitive when conducting your hypothesis testing (i.e. because your confidence interval at standard levels of significance is huge, you are more likely to fail to reject your null hypothesis), and also much more sensitive when you have bias in your sample. 

"Coming back to a more 'serious' level.... Given all that, I find it a little hard to comprehend how and why you'd think that their critique of the Lancet's statistical methodology would be any more credible than, say, Tickle-me Elmo (arf!.. arf-arf!... ooops, sorry)."
If their observation is correct, then why should it be disqualified.

========
Right you are. 

I just find it funny that a group so notorious for their comically inadecuate statistical methodologies would be cited as source of insightful questions regarding statistics.  However, you are right: address the question, not the questioner. Even if it *is* pretty comical.
==============

"Regarding you quotes - the one you posted was not from the Lancet, but from Iraqbodycount.org's own text in their "rebutal".  Which is great, but not particularly informative of what the Lancet published."

You see, I quoted what IBC said and provided reference. Then you I said where they got the number (in The Lancet). That was obviously not specific enough for you.

"Regarding your reference to the Lancet's study, well, hum...it's not in page 12 of the Lancet Survey, as you claim. Correct me if I'm wrong, but I think we're both reading the same document, and Page 12 contains the "Limitations" chapter and half of the "Conclusions" chapter.  Neither one of those even mentions the words "male" or "female", let alone make the claims that you state they do."

Take a look at the graph on p.12. Here is the quote, also p.12:

"In the next graph is shown the deaths from violent causes by age and sex. As can be seen, violent deaths account for most of the deaths, and violent deaths are almost entirely in males. Among the males, there were no practical survey methods to determine which of the deaths were among active combatants. It is interesting to note that the largest single age group of female deaths was among the under age 15 years."

"It's possible that it's elsewhere on the Lancet's survey - if you find it please let me know."

Done.


Ah, ok!   You're refering to Page 11 which contains a couple of graphs, not Page 12.  I suspect the difference is that you were counting the cover, while I was goign by the Study's page numbering. No big deal. 

Regading the figures, I did not see them before, but looking at your reference, it seems to me that the most plausible explanation is that some enterprising soul at Iraqbodycount.org took a ruler to the Page 11 graphs, and came out with the numbers you're refering to.


"In the meantime, I dont follow how you've concluded that a population that has suffered 2.5% mortality could *possibly* have a 7% mortality amongt males, even if every single casualty was male (which they were not).  Unless that happened *and* the population was overwhlmingly female (which it isnt)."

The other way round. Adults constitute less than 66% of pop. Of this group 50% are males. So this group comprisese less than 33% of entire pop. However, the survey found almost all violent deaths were suffered in this group. So

33%*7.5% = 2.5% = Lancet claim of nat'l average.

The actual numbers are

"Of the 287 violent post-invasion deaths recorded by the Lancet authors where the age and sex was known, 235 (82%) were adult males between 15 and 59 years old. Extrapolating to the population as a whole would mean that around 470,000 men in this age group have been killed violently, i.e. one in 15 (7%) of adult males aged 15 to 59"

link target="_blank">link even without the 60+ segment (as I read it).

So the numbers are consistent either way.

7.5% amongst adult males, or more than 7% according to IBC.

"The only way I can see around that is if you only count males in demographic groups where the impact of the war is particularly severe (of a particular age group, of a particular ethnic group, living in a particular location, etc..)  In which case, that claim has no meaning.  If you cherry-pick like that *of course* you're goign to find populations that have a much higher death rate.  Gee whiz, no wonder.

And in no way is that a disqualification of the Lancet's results - of course armed conflict afects certain demographic groups more heavily. For instance: military age males in places where the fighting is hardest tend to get the short end of the stick more often than 50-year olds living in peacefull provinces. That is, quite simply, the nature of warfare - dunno about the guitarists and retired librarians at Iraqbodycount.org, but I'd be truly amazed if anyone here would find *that* to be a surprising result...

Dont want to put words in your mouth, so I'd love to learn more about what this surprising result is and how you came to it."

Of course you wouldn't. I was just responding to this:

"- The Lancet survey does *not* indicate that 7% of the male population of Iraq has died over 3+ years of armed conflict.   The good folk at Iraqbodycount just made that one up.
Let's "reality-check" the math: since the Lancet survey found that 2.5% of the overall population of Iraq died over the course of the war, even if every single casualty was male that still does not account for 7% of the males. Oops!"

Anyway, it makes it possible to discuss if 1:10 to 1:15 are realistic numbers for mortality.

Ok - you *are* cherry picking. 

We're not talking about "males"; we're talking about "males between 15 and 59 who died a violent death".  In that case, I have no problem with the figure as such - if you cherry pick the segments with the highest mortality, you can easily get to that.  Above it, too.

However,  it's worth pointing out that you're buying into IBC's false assumption that violent deaths are the only conflict-related deaths, and are accepting numbers extrapolated from that flawed basis.

Naturally, they're not - and excess deaths from causes other than violence (such as privations, lack of medicine, heatstroke etc.) tend to strike the very old and the very young disproportionatelly. That reality was cheerfully ignored by the IBC team - they just saw that big "violent deaths" chart, took a ruler to it, and extrapolated to their heart's content.

When you think of it, it's not surprising; IBCs own methodology relies exclusively on newspaper reports of violent deaths. Further, that sort of extreme mathematical sloppiness is not unheard-of coming from a guitarist, a psychologist and "freelance researcher" for Greenpeace (the 3 guys who were the material authors of that particular pearl of wisdom). 

The upside is that it makes it a lot easier to catch - you dont have to worry about any sort of elaborate deception; just think "how could you screw this up at the most sophomoric level?" and run with that.

Heart,

eu4ea


Cheers

And now it is 3pm, erk!

Ok - you *are* cherry picking. 

We're not talking about "males"; we're talking about "males between 15 and 59 who died a violent death".  In that case, I have no problem with the figure as such - if you cherry pick the segments with the highest mortality, you can easily get to that.  Above it, too.

However,  it's worth pointing out that you're buying into IBC's false assumption that violent deaths are the only conflict-related deaths, and are accepting numbers extrapolated from that flawed basis.

Naturally, they're not - and excess deaths from causes other than violence (such as privations, lack of medicine, heatstroke etc.) tend to strike the very old and the very young disproportionatelly. That reality was cheerfully ignored by the IBC team - they just saw that big "violent deaths" chart, took a ruler to it, and extrapolated to their heart's content.

When you think of it, it's not surprising; IBCs own methodology relies exclusively on newspaper reports of violent deaths. Further, that sort of extreme mathematical sloppiness is not unheard-of coming from a guitarist, a psychologist and "freelance researcher" for Greenpeace (the 3 guys who were the material authors of that particular pearl of wisdom). 

The upside is that it makes it a lot easier to catch - you dont have to worry about any sort of elaborate deception; just think "how could you screw this up at the most sophomoric level?" and run with that.

Heart,

eu4ea

Sure, no reason to get grumpy.

 

"Nope.  Sorry. Flawed speculation based on unsupported conjecture. Not to be passed off as "according to the Lancet"."

 

Problem here is that it is not conjecture as it is consistent with Lancet, but a little error. So it is 22 instead of 11. 7% instead of 3.5. Rough numbers to give you a sense of it. Also demonstrates statistics: Change crude death rate with 0.2 and you're there. (And curiously I have found out they used 5.4 and not 5.5 for CDR!)

 

ANyway, really touching on substance here. ;)

 

 

"My suggestion would be to stop messing around with the absurd math proposed by the statistical geniuses at IBC, and... read the dang survey. Page 7, on the bottom. Where it says "53,938 excess deaths caused by non-violent causes"."

 

I'll accept that number. However, IBC still only concerned with violent deaths. Still missing 600,000. The 7 % figure hasn't been compromised! Hmmm.

 

 

"What bugs me about this is not the fact that you're off by a factor of 2 - it's that you've brought off on the mathematical inanity of the diletantes at IBC.  Those guys may have a real shaky grasp on the notion of a war related non-violent death, or a non-war related violent deaths - but there's no good reason why we should follow them down that rabbit trail. It's, well, dumb."

 

Not"

 

My error of calc is unrelated with what IBC said, but was related to your attempt to push the violent deaths over in the non-violent group. To do that would be, well, disingeneous. But if you insist on direct Lancet quotes, here is one from the actual article as it appears in Lancet:

 

"Post-invasion excess mortality rates showed much the

same escalating trend, rising from 2·6 per 1000 people

per year (0·6–4·7) above the baseline rate in 2003 to

14·2 per 1000 people per year (8·6–21·5) in 2006

(fi gure 2 and table 3). Excess mortality is attributed

mainly to an increase in the violent death rate; however,

an increase in the non-violent death rate was noted in

the later part of the post-invasion period (2005–06). The

post-invasion non-violent excess mortality rate was

0·7 per 1000 people per year (–1·2 to 3·0)."

 

Yup. That was 0.7. Directly from Lancet.

 

Here is what it say in the doc that circulates here and the proofread, corrected article:

 

"Violent death rates

As there were few violent deaths in the survey population prior to the invasion, all violent deaths can be considered “violent excess deaths.” The post-invasion violent death rate was:

• March 2003-April 2004: 3.2 deaths/1000/year

• May 2004-May 2005: 6.6 deaths/1000/year

• June 2005-June 2006: 12.0 deaths/1000/year

• Overall post-invasion: 7.2 deaths/1000/year"

 

I guess we can agree that 0.7/7.2 = 93% of excess deaths are from violence. Right?

 

And it has also been established that according to Lancet they were adult males, right? Ok another quote:

 

"As can be seen, violent deaths account for most of the deaths, and violent deaths are almost entirely in males. Among the males, there were no practical survey methods to determine which of the deaths were among active combatants. It is interesting to note that the largest single age group of female deaths was among the under age 15 years."

 

So, they (BC)  are actually right. 7% + in the adult male group.

 

How would you do it?

 

"Look, TAC, I really dont want to get grumpy about this but... no. 

Not by a long shot; I'm amazed that anyone with a knowledge of military history would argue that.  Here are the figures:

 

Those figures are casualty rates as a percentage of those enlisted, not just people who saw "trench warfare".  Even then, notice not a single one of the mayor participants had a casualty rate under 7% - in fact many had double that. Further, in most cases you can take most of the "Missing/POW" column and add it to the 'dead' column; most of those guys never made it home."

 

I knew the figures for GB, so 7% was pretty close to 10.2%. Didn't realize that the others were significantly higher. But wait! WWI lasted 4.16yrs vs 3.3 for Iraq! That makes it 8.1% for GB in WW1 vs 7% + for Iraq. Less than 1% in difference?

 

Also, GB suffered 750,000 dead out of a pop of 46,000,000. That's 1.6% in 4.16 yrs of war. Lancet says 2.5% for 3.3 in Iraq.

 

Make another of these surveys in a year and we are talking France...

 

 

"I guess what gets my goat about the whole thing (aside from the hoky-poky math of IBC) is the grandiloquence of the claims - worse the Hiroshima! bloodier than the Western Front!. Which it  isnt - it's not epically bloody; it's a nasty little war  in a former colony with at least four armed groups vying for supremacy."

 

Lancet et al are also the problem here. They are providing crap material for those Hiroshima type arguments.

 

It is easy and benign enough for us to sit here and discuss/argue (whatever) this. At least IBC provides a minimum figure (lacks completeness). But producing blue sky numbers like Lancet does pose a problem.

 

 

My opinion on this, is that we are actually talking October surprises. Does it bugger me on internal US issues. No. I'm European, I personally don't care who gets this or that house. And if I should vote, I would probably vote Democrat or whatever candidate I trusted. Leave that be.

 

So, some people are blinded by their enemy, Bush/US/neocons/etc, and don't care what methods are used. And they certainly don't have an eye for sideeffects of their methods. Again do I care? Well, yes, because all that unsubstantiated crap floating around makes an impact outside of US elections or general antiwar campaigning.

 

An illusion is being created for mullahs to use to antagonize. Writers of sh!tlists listing how many people America has killed in wars cumulated from the War of Independence till today will have a heyday. Need I go on?

 

The damage it does is in the enforcing of the culture of grievance and victimization across the ME and muslim world, when the discussions are going in the teahouses etc...

 

And when this is the case, you better have the FACTS, and not something that presents itself as facts and builds itself on so many assumptions and has failure points. It HAS to be correct and of MUCH better quality.

 

392,979 – 942,636 excess deaths. Go pick a number - it's a fact.

 

That is the issue I have with this report.

 

Cheers

Here's the final nail in the coffin for me on this most recent Roberts et al study.  No attempt to even benchmark results against the last detailed census, dismissing portions of his own report because it was written by a student (ever heard of personal responsibility?  hello?  your name is on the study!), and he isn't even willing to try and defend his study against other professionals in the field. 

 

The "discredited" title for the study fits very well.

 

***

Wall Street Journal
October 18, 2006
Pg. 20

Shek

 

Thanks for posting. Ive seen that this morning all over the place.

 

THAT was the missing piece on cluster studies that I couldnt articulate earlier but made me uneasy enough to question the study. (Must be getting old - in my earlier days, I wouldnt have missed the obviousness of the low number of clusters extended to the 22 million population, whereas higher clusters for even lower pops were conducted in other countries)

 

Cluster samplings by nature are inherently sensitive to low numbers and outliers, accounting for higher Variance in the results, which is why you could have positive growth in Iraqis of 100,000 to almost 1,000,000 dead. The higher the number of clusters, the lower the Variance, and therefore the more confidence one has in the results.

 

And that they didnt even look at demographic to ensure a representative sample as well? Condemns the research as fatally flawed unless that could be explained as well.

 

swhitebull - it will be interesting to see how EU spins this

 

swhitebull