Skip to content or view mobile version

Home | Mobile | Editorial | Mission | Privacy | About | Contact | Help | Security | Support

A network of individuals, independent and alternative media activists and organisations, offering grassroots, non-corporate, non-commercial coverage of important social and political issues.

The Government's misleading response to the Lancet Iraq Mortality Survey

shocked and awed | 08.11.2004 16:16 | Analysis | Anti-militarism | Health | Cambridge | London

The Government's response to the Iraq Mortality Survey published last week in leading public health journal The Lancet (estimating that around 100,000 excess deaths may have been caused by the invasion of Iraq) is deeply inaccurate and misleading. Amongst other things it constitutes an opportunistic political attack on academic standards.

Cambridge Solidarity with Iraq: Briefing Note 04/11/04

THE LANCET IRAQ MORTALITY SURVEY: THE UK GOVERNMENT’S RESPONSE IS
INACCURATE AND MISLEADING

On Friday 29th October 2004, the Lancet medical journal published a study
of post-war mortality in Iraq which estimated that at least 100,000 excess
civilian deaths had occurred since the 2003 invasion; that most were caused
by violence; and that most of those violent deaths were caused by coalition
air strikes.

In his daily press briefing that morning, the Prime Minister's Official
Spokesperson (PMOS) dismissed the study since its methodology was, he
claimed, inappropriate:

“Asked if the Prime Minister was concerned about a survey published today
suggesting that 100,000 Iraqi civilians had died as a result of the war in
Iraq, the PMOS said that it was important to treat the figures with caution
because there were a number of concerns and doubts about the methodology
that had been used. Firstly, the survey appeared to be based on an
extrapolation technique rather than a detailed body count. Our worries
centred on the fact that the technique in question appeared to treat Iraq
as if every area was one and the same. In terms of the level of conflict,
that was definitely not the case. Secondly, the survey appeared to assume
that bombing had taken place throughout Iraq. Again, that was not true. It
had been focussed primarily on areas such as Fallujah. Consequently, we did
not believe that extrapolation was an appropriate technique to use.”
( http://www.number-10.gov.uk/output/Page6535.asp)

These criticisms of the Lancet mortality survey were repeated again on 1st
November. They are misleading, and almost entirely unfounded. This briefing
note examines the claims of the Prime Minister’s Official Spokesperson.

1) "Firstly, the survey appeared to be based on an extrapolation technique
rather than a detailed body count."

The fact that the survey uses an extrapolation technique does not
automatically mean that it is less likely to be accurate than a body count.
In fact, it is more likely to be accurate than existing attempts at body
counts of Iraqi civilians. This is because in Iraq, where there are so many
no-go areas, it would be impossible to count every casualty.

Attempts to do body counts through deaths reported in the press,
like Iraq Body Count, are also necessarily underestimates, since press
reports of casualties will be incomplete, not least because the areas where
people are being killed are the same areas into which journalists don't
dare go. As Iraq Body Count states on its website:

"Our maximum therefore refers to reported deaths - which can only be a
sample of true deaths unless one assumes that every civilian death has been
reported. It is likely that many if not most civilian casualties will go
unreported by the media." (  http://iraqbodycount.net/#position )

In these circumstances, population-based research like the Lancet study is
arguably a better source.

A detailed, comprehensive body count on the ground by military and medical
personnel would perhaps be more accurate, but it is a task which the US and
UK forces in Iraq refuse to carry out. For Downing Street to question the
best available study on the grounds of preferring a methodology which they
themselves refuse to operate is surprising, to say the least.

2) "Our worries centred on the fact that the technique in question appeared
to treat Iraq as if every area was one and the same."

The technique does not treat Iraq as if every area was the same. The survey
takes a sample in each of 33 areas throughout Iraq’s 18 governorates,
precisely so that regional variation is taken into account as much as
possible. All regions of the country were sampled.

The PMOS might be referring to the fact that due to the deteriorating
security situation in Iraq, the survey 'paired' 12 of the 18 governorates,
and took samples from only 1 of the pair in order to reduce danger to the
investigators by reducing the necessary travel. In these cases, however,
the process of assigning samples to one of the two governorates was
randomised, and weighted according to the population of the governorates,
thus 'still sampling from all regions of the country' and ensuring that
'the sample remained a random national sample' (Roberts et al p.2)

The PMOS may also be referring to the fact that the sample was not
stratified: each of the cluster samples contributed equally to the national
estimate. In this sense they were considered 'exchangeable' for the
purposes of analysis. Where the characteristics of a population are better
known, some surveys use a 'stratified' sample, whose composition is
weighted according to the known characteristics of the population e.g. they
contain the same proportion of men/women, social classes, proportion of
groups subject to violence etc. etc. In this case, the number of samples
attributed to each governorate was weighted according to the governorate's
population; but beyond that the samples were not stratified according, for
instance, to regional differences in exposure to violence, for the
methodologically sound reason that the distribution of violence in Iraq is
simply not known. As Dr Les Roberts, the lead author of the Lancet report,
stated in response to a query from Medialens (quoted at
www.medialens.org/blog ):

"We would have had a more accurate picture if we conducted a 'stratified'
sample, with some in the high violence areas and some in the low violence
areas. But, that would have involved visiting far more houses and exposing
the interviewers to even more risk. Secondly, we do not know how many
people are in the 'high violence' areas, so this would have involved large
assumptions that would now be criticized."

Importantly, though, this does NOT mean that all areas were treated as the
same, in the larger sense of extrapolating from a single area to the entire
national population, as the PMOS is perhaps trying to suggest. All areas of
Iraq were sampled to try to reflect regional variations as much as was
possible given the limitations of current knowledge and safety.

3) "Secondly, the survey appeared to assume that bombing had taken place
throughout Iraq. Again, that was not true. It had been focussed primarily
on areas such as Fallujah."

This is also untrue - see above. The survey was designed precisely to take
variations throughout Iraq into account. Moreover, since the mortality rate
due to violent deaths in the sample collected in Fallujah was so abnormally
high, the Fallujah sample was excluded from the study’s final estimate.
Thus far from overestimating the deaths caused elsewhere by extrapolating
them from those clustered in Fallujah, as the PMOS suggests, in fact the
figure of 100,000 deaths does not take into account the exceptionally high
mortality probably caused in the area of the most intensive bombing, namely
Fallujah. Had the Fallujah sample been included, the survey's estimate
would have been of an excess of about 298,000 deaths, with 200,000
concentrated in the 3% of Iraq around Fallujah (Roberts et al p.5).

In addition, contrary to the PMOS’s claims about the tight focus of
bombing, the Lancet study in fact found that in areas well away from
Fallujah the most significant cause of the deaths it recorded was still
violence, most attributed to coalition bombing (Roberts et al p.4).

4) "Consequently, we did not believe that extrapolation was an appropriate
technique to use."

Almost all surveys except censuses involve 'extrapolation techniques'
(using a sample rather than the entire population), since it is simply
impossible to survey every member of a country's population. Extrapolation
is a universally recognised method that governments and academics use to
obtain data, and it is remarkable to see a government spokesperson
rejecting it outright.

In addition, the specific sampling technique used in the Iraq mortality
survey, a clustered sample survey, is widely used by government and other
surveys to gather data from geographical clusters of households. See, for
example, the recent report from the Home Office Research, Development and
Statistics Directorate, 'Asylum seekers' experiences of the voucher scheme
in the UK, March 2002', which used cluster samples in order to reduce the
cost of the survey (  http://www.homeoffice.gov.uk/rds/pdfs2/asylumexp.pdf
- see pp.2-3 of this report).

5) Iraqi Ministry of Health Figures

(a) PMOS morning briefing, 1st November: "The Iraqi Department of Health
had issued figures that showed over a 6 month period there were about 3000
deaths, which was a long way short of the figures quoted in the "Lancet".
The Iraqi DOH measured those figures by the number of people who came into
hospitals throughout Iraq, and it was very difficult to rely on any such
figures quoted in the "Lancet" with any certainty."
( http://www.number-10.gov.uk/output/page6515.asp )

(b) Prime Minister, House of Commons, 4th November: "we do not accept the
figures released by The Lancet last week at all. The Iraqi Ministry of
Health has put out figures for the six months up to October, which suggest
just over 3,000 deaths, but that includes people who are either terrorists
or insurgents and those who have been the victims of terrorist attack."
( http://www.publications.parliament.uk/pa/cm200304/cmhansrd/cm041103/debtext/41103-03.htm#column_301 )

Like figures taken from Press Reports like Iraq Body Count, figures
obtained by counting the numbers of bodies arriving at hospitals, will also
necessarily be undercounts: many of those who have died or been killed will
never arrive at hospital in conditions of war, when access to roads and
health facilities are severely disrupted, and when it might seem pointless
to risk the journey for the sake of someone who is already dead.

It is unclear where the MOH figures referred to by the PM were released.
If, however, he is referring to the MOH figures obtained by Knight Ridder
Newspapers of "3,487 Iraqi deaths in 15 of the country's 18 provinces from
April 5 [i.e. not including the 3 Kurdish Governorates] - when the ministry
began compiling the data - until Sept. 19", then he is being misleading.
The MOH officials interviewed by Knight Ridder EXPLICITLY STATED THAT THEIR
FIGURES WERE AN UNDERCOUNT:

"Iraqi health and hospital officials agreed that the statistics captured
only part of the death toll. To compile the data, the Health Ministry calls
the directors general of the 15 provinces and asks how many deaths related
to the war were reported at hospitals. The tracking of such information has
become decentralized since the fall of Saddam Hussein's regime because both
hospitals and morgues issue death certificates now. And families often bury
their dead without telling any government agencies or are treated at
facilities that don't report to the government....The numbers also exclude
those whose bodies were too mutilated to be recovered at car bombings or
other attacks, the ministry said. Ministry officials said they didn't know
how big the undercount was. "We have nothing to do with politics," Jassim
said." ( http://www.realcities.com/mld/krwashington/9753603.htm )

The trends reported in the MOH figures also support some of the Lancet
study's findings: the MOH found that substantial numbers of women and
children were being killed by coalition violence (328 of their 3487
figure); and that more (two-thirds) were being killed by coalition than by
insurgent attacks (although the reliability of this latter distinction is
unclear).

***

Perhaps more serious than these misrepresentations of the mortality study
by the Prime Minister's Office is the UK Government's apparent disregard
for efforts to ascertain the numbers of civilian deaths its actions have
caused. Foreign Office Minister Baroness Symons promised the House of Lords
that:

“we have done our best to count them [civilian casualties] but I am bound
to say to the noble Baroness that the practicalities involved are very
difficult indeed. Certainly during the early part of the engagement, it has
not always been possible to know what civilian casualties there were on the
ground….We do our best to count civilian casualties now."

(Lords Hansard 27 Apr 2004: Column 686)

Yet neither the US nor the UK governments have made public any aggregate
data on civilian deaths in Iraq – a failure which an MOD spokesperson
defended by arguing that

“there is no reliable way of estimating the number of civilian casualties
caused during major combat operations.”

(BBC Online 22 September 2004)

The Lancet mortality survey argues that this is not the case:

“This survey shows that with modest funds, 4 weeks, and seven Iraqi team
members willing to risk their lives, a useful measure of civilian deaths
could be obtained. There seems to be little excuse for occupying forces to
not be able to provide more precise tallies.”

(Roberts et al p.7)

In the last few days the Government have attempted to disparage the Lancet
report on a different tack, by going back on its previous rejections of
casualty estimates and instead positively citing the much lower estimate of
the Iraq Body Count project, as Foreign Secretary Jack Straw did when
interviewed on the Radio 4 ‘Today’ programme on 29th October. Mr Straw
ignores the fact that Iraq Body Count itself makes clear that its figures
cannot be considered a comprehensive estimate of civilian casualties, since
‘it is likely that many if not most civilian casualties will go unreported
by the media.’ (see above). The Government’s new attention to the figures
provided by this media-report-based casualty estimate is also surprising
given the previous statement by an MOD spokesperson about Iraq Body Count
that

“ We would caution against taking the numbers quoted in media reports and
else where at face value. No source or combination of sources can produce a
reliable figure."

(BBC Online 22 September 2004)

and given Defence Minister Adam Ingram’s dismissive answer to a
parliamentary question in June 2003 asking “what information he has (a)
given to and (b) received from the organisation called Iraq Body Count”, Mr
Ingram replying simply that “Defence Ministers have had no contact with an
organisation called Iraq Body Count.” (Commons Written Answers, 3 Jun 2003
Col. 37W, question 114211)

The Government must accept that despite shortcomings due to the
difficulties faced by a small team of volunteer doctors collecting data in
a war zone, the Lancet report is the most comprehensive study of Iraqi
civilian deaths since 2003 currently available. If it is to rebuff its
findings, the Government must honour its own obligations by undertaking a
large-scale survey of civilian casualties. As the Lancet report states:

“ US General Tommy Franks is widely quoted as saying “we don’t do body
counts”. The Geneva Conventions have clear guidance about the
responsibilities of occupying armies to the civilian population they
control. The fact that more than half the deaths reportedly caused by the
occupying forces were women and children is cause for concern. In
particular, Convention IV, Article 27 states that protected persons “. . .
shall be at all times humanely treated, and shall be protected especially
against acts of violence . . .”. It seems difficult to understand how a
military force could monitor the extent to which civilians are protected
against violence without systematically doing body counts or at least
looking at the kinds of casualties they induce. In view of the political
importance of this conflict, these results should be confirmed by an
independent body such as the ICRC, Epicentre, or WHO.”

(Roberts et al p.7)


FURTHER INFORMATION:

1) L. Roberts, G. Burnham, R. Lafta, J. Khudhairi, R.
Garfield, ‘Mortality before and after the 2003 invasion of Iraq: cluster
sample survey’,  http://image.thelancet.com/extras/04art10342web.pdf


2) Iraq Body Count, the most comprehensive indirect casualty count:
 http://iraqbodycount.net/ This site includes a summary of another casualty count project, and an explanation of IBC methodology.

Author: Michael Lewis, Christ's College, Cambridge CB2 3BU, United Kingdom
 mhl24@cam.ac.uk

shocked and awed

Upcoming Coverage
View and post events
Upcoming Events UK
24th October, London: 2015 London Anarchist Bookfair
2nd - 8th November: Wrexham, Wales, UK & Everywhere: Week of Action Against the North Wales Prison & the Prison Industrial Complex. Cymraeg: Wythnos o Weithredu yn Erbyn Carchar Gogledd Cymru

Ongoing UK
Every Tuesday 6pm-8pm, Yorkshire: Demo/vigil at NSA/NRO Menwith Hill US Spy Base More info: CAAB.

Every Tuesday, UK & worldwide: Counter Terror Tuesdays. Call the US Embassy nearest to you to protest Obama's Terror Tuesdays. More info here

Every day, London: Vigil for Julian Assange outside Ecuadorian Embassy

Parliament Sq Protest: see topic page
Ongoing Global
Rossport, Ireland: see topic page
Israel-Palestine: Israel Indymedia | Palestine Indymedia
Oaxaca: Chiapas Indymedia
Regions
All Regions
Birmingham
Cambridge
Liverpool
London
Oxford
Sheffield
South Coast
Wales
World
Other Local IMCs
Bristol/South West
Nottingham
Scotland
Social Media
You can follow @ukindymedia on indy.im and Twitter. We are working on a Twitter policy. We do not use Facebook, and advise you not to either.
Support Us
We need help paying the bills for hosting this site, please consider supporting us financially.
Other Media Projects
Schnews
Dissident Island Radio
Corporate Watch
Media Lens
VisionOnTV
Earth First! Action Update
Earth First! Action Reports
Topics
All Topics
Afghanistan
Analysis
Animal Liberation
Anti-Nuclear
Anti-militarism
Anti-racism
Bio-technology
Climate Chaos
Culture
Ecology
Education
Energy Crisis
Fracking
Free Spaces
Gender
Globalisation
Health
History
Indymedia
Iraq
Migration
Ocean Defence
Other Press
Palestine
Policing
Public sector cuts
Repression
Social Struggles
Technology
Terror War
Workers' Movements
Zapatista
Major Reports
NATO 2014
G8 2013
Workfare
2011 Census Resistance
Occupy Everywhere
August Riots
Dale Farm
J30 Strike
Flotilla to Gaza
Mayday 2010
Tar Sands
G20 London Summit
University Occupations for Gaza
Guantanamo
Indymedia Server Seizure
COP15 Climate Summit 2009
Carmel Agrexco
G8 Japan 2008
SHAC
Stop Sequani
Stop RWB
Climate Camp 2008
Oaxaca Uprising
Rossport Solidarity
Smash EDO
SOCPA
Past Major Reports
Encrypted Page
You are viewing this page using an encrypted connection. If you bookmark this page or send its address in an email you might want to use the un-encrypted address of this page.
If you recieved a warning about an untrusted root certificate please install the CAcert root certificate, for more information see the security page.

Global IMC Network


www.indymedia.org

Projects
print
radio
satellite tv
video

Africa

Europe
antwerpen
armenia
athens
austria
barcelona
belarus
belgium
belgrade
brussels
bulgaria
calabria
croatia
cyprus
emilia-romagna
estrecho / madiaq
galiza
germany
grenoble
hungary
ireland
istanbul
italy
la plana
liege
liguria
lille
linksunten
lombardia
madrid
malta
marseille
nantes
napoli
netherlands
northern england
nottingham imc
paris/île-de-france
patras
piemonte
poland
portugal
roma
romania
russia
sardegna
scotland
sverige
switzerland
torun
toscana
ukraine
united kingdom
valencia

Latin America
argentina
bolivia
chiapas
chile
chile sur
cmi brasil
cmi sucre
colombia
ecuador
mexico
peru
puerto rico
qollasuyu
rosario
santiago
tijuana
uruguay
valparaiso
venezuela

Oceania
aotearoa
brisbane
burma
darwin
jakarta
manila
melbourne
perth
qc
sydney

South Asia
india


United States
arizona
arkansas
asheville
atlanta
Austin
binghamton
boston
buffalo
chicago
cleveland
colorado
columbus
dc
hawaii
houston
hudson mohawk
kansas city
la
madison
maine
miami
michigan
milwaukee
minneapolis/st. paul
new hampshire
new jersey
new mexico
new orleans
north carolina
north texas
nyc
oklahoma
philadelphia
pittsburgh
portland
richmond
rochester
rogue valley
saint louis
san diego
san francisco
san francisco bay area
santa barbara
santa cruz, ca
sarasota
seattle
tampa bay
united states
urbana-champaign
vermont
western mass
worcester

West Asia
Armenia
Beirut
Israel
Palestine

Topics
biotech

Process
fbi/legal updates
mailing lists
process & imc docs
tech