<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=iso-8859-1" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.6001.18939">
<STYLE></STYLE>
</HEAD>
<BODY bgColor=#ffffff>
<DIV><FONT size=2></FONT> </DIV><FONT size=2>
<DIV><BR>
<DIV id=pgtop class=hldpg>
<H1>How DNA evidence creates victims of chance </H1>
<UL class=markerlist>
<LI>18 August 2010 by <A
href="http://www.newscientist.com/search?rbauthors=Linda+Geddes"><B>Linda
Geddes</B></A>
<LI>Magazine issue <A href="http://www.newscientist.com/issue/2774">2774</A>.
<A href="http://www.newscientist.com/subscribe?promcode=nsarttop"><B>Subscribe
and save</B></A>
<LI>For similar stories, visit the <A
href="http://www.newscientist.com/topic/crime-and-forensics"><B>Crime and
Forensics</B></A> and <A
href="http://www.newscientist.com/topic/genetics"><B>Genetics</B></A> Topic
Guides </LI></UL></DIV><!-- pgtop -->
<DIV id=hldmain class="hldpg floatclearfix">
<DIV id=hldcontent class=floatleft>
<DIV id=maincol class=floatleft>
<DIV id=artImg><IMG
title="What are the odds that he deserves the cuffs? (Image: Erika Kyte/Photonica/Getty)"
alt="What are the odds that he deserves the cuffs? (Image: Erika Kyte/Photonica/Getty)"
src="http://www.newscientist.com/data/images/ns/cms/mg20727743.300/mg20727743.300-2_300.jpg">
<P class=lowlight>What are the odds that he deserves the cuffs? (Image: Erika
Kyte/Photonica/Getty)</P>
<P class=marker><A
href="http://www.newscientist.com/articleimages/mg20727743.300/1-how-dna-evidence-creates-victims-of-chance.html"
target=ns>2 more images</A></P></DIV>
<P class=infuse><B>Read more:</B> <I><A
href="http://www.newscientist.com/article/mg20727733.500-fallible-dna-evidence-can-mean-prison-or-freedom.html">Fallible
DNA evidence can mean prison or freedom</A></I></P>
<P class=infuse>CHARLES RICHARD SMITH has learned the hard way that you can
prove almost anything with statistics. In 2009 a disputed statistic provided by
a DNA analyst landed him with a 25-year jail sentence.</P>
<P class=infuse>Smith was convicted of a sexual assault on Mary Jackson (not her
real name) in Sacramento, California, which took place in January 2006. Jackson
was sitting in a parking lot when a stranger jumped into her truck and made her
drive to a remote location before forcing her to perform oral sex on him. When
police arrested Smith and took a swab of cells from his penis, they found a
second person's DNA mixed with his own.</P>
<P class=infuse>The DNA analyst who testified in Smith's trial said the chances
of the DNA coming from someone other than Jackson were 1 in 95,000. But both the
prosecution and the analyst's supervisor said the odds were more like 1 in 47. A
later review of the evidence suggested that the chances of the second person's
DNA coming from someone other than Jackson were closer to 1 in 13, while a
different statistical method said the chance of seeing this evidence if the DNA
came from Jackson is only twice that of the chance of seeing it if it came from
someone else.</P>
<P class=infuse>How can a single piece of DNA evidence generate such massive
differences in the statistical weight assigned to it? Last week, a <I>New
Scientist</I> investigation showed how <A
href="http://www.newscientist.com/article/mg20727733.500-fallible-dna-evidence-can-mean-prison-or-freedom.html">different
forensic analysts can reach very different conclusions</A> about whether or not
someone's DNA matches a profile from a crime scene. This week we show how, even
when analysts agree that someone could be a match for a piece of DNA evidence,
the statistical weight assigned to that match can vary enormously.</P>
<P class=infuse>"Usually DNA evidence is pretty strong," says <A
href="http://www.zebfontaine.eclipse.co.uk/djb.htm" target=nsarticle>David
Balding</A>, a statistical geneticist at University College London, whose
calculation puts the lowest probability on the link between Smith and Jackson.
"My point is that the number juries are provided with often overstates the
evidence. It should be a smaller number."</P>
<DIV class="quotebx bxbg">
<DIV class=quoteopen>
<DIV class=quoteclose>
<DIV class="quotebody lowlight"><QUOTE><QUOTETEXT>The odds juries are provided
with often overstate the evidence</QUOTETEXT></QUOTE> </DIV></DIV></DIV></DIV>
<P class=infuse>There are several types of statistic that analysts can attach to
DNA evidence. In basic cases involving a large amount of DNA from a single
person, you can simply calculate how common their profile is in the general
population- this is called the random match probability (RMP). However, the RMP
becomes problematic when looking at mixed or degraded samples of DNA, where part
of a person's DNA profile may be missing or hidden by another person's DNA.</P>
<P class=infuse>For this reason many labs will use a different statistic when
interpreting mixtures, such as "random man not excluded" (RMNE) or the "combined
probability of inclusion or exclusion" (CPI/E). These calculate the odds that
DNA in a mixture is from a random person rather than the person you're
interested in.</P>
<P class=infuse>But this approach by no means solves the problems. In Smith's
case, two of the statistics given- 1/95,000 and 1/47- were the result of RMNE or
CPI calculations, while the 1/13 statistic was a variation on these.</P>
<P class=infuse>A DNA profile consists of a series of peaks relating to specific
locations on the chromosomes, called loci. In a standard profile there should be
peaks indicating two genetic sequences, or alleles, at every locus- one from
each parent. However, in mixed profiles or when only small amounts of DNA are
present, it can be difficult to work out which alleles came from whom, and even
to detect whether certain alleles are present (<I>New Scientist</I>, 14 August,
p 8).</P>
<P class=infuse>In the Smith case, the sample containing another person's DNA
showed alleles at seven out of a possible 15 loci, but at four of these loci,
the alleles matched those of both the victim and the defendant. "The 1 in 95,000
figure in effect treated these alleles as full-weight evidence that the DNA came
from the victim, ignoring the alternative possibility that the allele we saw
could have been from the defendant," says Balding. If the opposite position is
taken, and these alleles are ignored, you come up with a figure closer to 1 in
13. "It's a question of which loci you consider," he says.</P>
<P class=infuse>At present there are no firm guidelines on which alleles should
be included in an RMNE calculation, meaning that different labs can come up with
very different statistics.</P>
<P class=infuse>Balding and <A
href="http://www.chem.strath.ac.uk/people/academic/peter_gill"
target=nsarticle>Peter Gill</A> of the University of Strathclyde in Glasgow, UK,
who is a former chief analyst at the UK's Forensic Science Service (FSS), have
developed a different statistic, the likelihood ratio (LR), which attempts to
take phenomena like missing alleles into account. "The likelihood ratio tries to
do the right thing," says Balding. "It asks how probable is this evidence under
the prosecution case, and how probable is this evidence under the defence case?
Then you take the ratio of the two."</P>
<H3 class=crosshead>Useless at trial</H3>
<P class=infuse>In the Smith case, Balding calculated an LR of just 2, meaning
that the DNA evidence is only twice as likely if it came from the victim as from
someone else. Since there are many alternative possibilities for the source of
the DNA, a 2 to 1 ratio means the evidence is so weak as to be virtually useless
at trial.</P>
<P class=infuse>Yet neither the LR of 2 nor the 1/13 statistic was presented in
court. Smith was ultimately convicted on the basis of this DNA and other
evidence. He is still in prison, and plans to appeal.</P>
<P class=infuse>Using a likelihood ratio doesn't necessarily mean that the
defendant will get an easier ride. In a 2003 murder case in the UK, Balding's LR
found that the evidence was 10 million times more likely to fit the
prosecution's case than that of the defence (<I>Forensic Science International:
Genetics</I>, <A href="http://dx.doi.org/10.1016/j.fsigen.2009.03.003"
target=nsarticle>DOI: 10.1016/j.fsigen.2009.03.003</A>).</P>
<P class=infuse>"All evidence is worthwhile, it's just a question of can you
evaluate it fairly?" says Balding.</P>
<P class=infuse>These cases are not isolated incidents. A recent study by <A
href="http://www.cstl.nist.gov/strbase/butler.htm" target=nsarticle>John
Butler</A> at the US National Institute of Standards and Technology (NIST) in
Gaithersburg, Maryland, found widespread variation in the statistics reported by
individual laboratories. When he gave the same DNA evidence to 69 different US
labs and asked them to provide conclusions about whether or not the suspect was
a match, some labs reported RMP, others reported RMNE, while others gave no
statistic at all. "There was a difference of about 10 orders of magnitude in
terms of the statistical results that were obtained on the same samples," says
Butler.</P>
<P class=infuse>Even among the labs using the same statistic- the RMNE- there
were differences in the figure they came up with, depending on which alleles
they chose to include or discard. What it could mean is that a jury presented
with evidence from one lab is told that the chance of a match is around 1 in
100,000, say, while a different lab might say it is 1 in a quadrillion
(10<SUP>15</SUP>).</P>
<H3 class=crosshead>Throwing evidence away</H3>
<P class=infuse>Our own survey of DNA laboratories backs up these conclusions
<FIGREF refid="mg27743301.jpg">(see chart)</FIGREF>. We asked 19 labs around the
world which statistics they report when dealing with complex DNA mixtures. Six
said that they reported RMP, six reported RMNE, while two said they reported LR.
Five labs said they either report no statistics or a CPI/E. The type of
statistic reported even varied within the same US state.</P>
<P class=infuse>The International Society for Forensic Genetics recently issued
guidelines recommending the use of the LR when dealing with complex mixtures.
Even so, few labs have taken it up. The Institute of Environmental Science and
Research lab in Auckland, New Zealand, which performs DNA analysis for the New
Zealand police, is one of the few to do so. "Likelihood ratios are more complex,
and they're harder to present in court," says John Buckleton, head of ESR's
biology lab. For this reason, introducing them can require training for judges
to ensure they can accurately explain the statistic to jurors.</P>
<P class=infuse>Even the UK's FSS, where Gill worked for 25 years, has not fully
implemented LRs. "They report LRs, but they're not necessarily incorporating
things like the probability of [missing alleles]," says Gill. "By not using the
proper statistical methods, you're just throwing evidence away." When <I>New
Scientist</I> contacted the FSS, we were told that it does evaluate DNA evidence
using LRs. "In addition," said a spokesperson, "we use the replication method of
analysis when dealing with low levels of DNA, which means that the scientist
reports the alleles that can be replicated."</P>
<P class=infuse>Earlier this year, the Scientific Working Group on DNA Analysis
Methods (SWGDAM), which provides guidance to US forensic labs, issued its own
recommendations regarding the use of statistics for DNA mixtures. It also
proposes LR as a suitable statistic, as well as providing stronger guidance on
how RMNE and RMP should be calculated. SWGDAM also insists that analysts must
provide a statistic whenever they claim that someone's DNA might be included in
a mixture. "There are some labs that are just reporting that they think it's a
match- in their opinion," says Butler. "That's a problem, because the jury says:
'Oh, it's DNA? It matches? Guilty.'"</P>
<DIV class="quotebx bxbg">
<DIV class=quoteopen>
<DIV class=quoteclose>
<DIV class="quotebody lowlight"><QUOTE><QUOTETEXT>Some labs give no statistics
on a match, just an opinion. That's a problem</QUOTETEXT></QUOTE>
</DIV></DIV></DIV></DIV>
<P class=infuse>Ultimately though, there will be cases where the DNA evidence is
so complex that it is impossible to generate a reliable statistic. Buckleton
admits that DNA analysts can come under pressure from prosecutors and police, as
well as defence lawyers, to render DNA evidence useable. "I'm not sure that the
pressure is to drive the numbers in any particular direction, but perhaps to be
more certain than you should be at some points," he says.</P>
<P class=infuse>In really complex cases, analysts need to be able to draw a line
and say "This is just too complex, I can't make the call on it," says Butler.
"Part of the challenge now, is that every lab has that line set at a different
place. But the honest thing to do as a scientist is to say: I'm not going to try
to get something that won't be reliable."</P>
<DIV class="artbx bxbg">
<H3 id=bx277433B1>When lawyers question DNA</H3>
<P>In 2007, Sean Hoey was cleared of involvement in the 1998 Omagh bombing in
Northern Ireland, which killed 29 people. At the heart of the case against him
was DNA on the bomb timers, which the prosecution alleged matched Hoey's. It was
the first time that defence lawyers had challenged DNA evidence in a UK court,
and they were successful, arguing that DNA analysts were divided over the
reliability of the technique used. In low-copy-number (LCN) testing, tiny
amounts of DNA are amplified to generate a profile.</P>
<P>Since then, much of the scrutiny surrounding DNA evidence has focused on the
reliability of LCN testing, but some feel that this emphasis is misplaced.</P>
<P>In the wake of the Hoey case, Belfast-based solicitor Peter Corrigan of Kevin
Winters and Company has routinely sought access to the lab reports behind the
DNA evidence presented in court, which has resulted in four successful
challenges. "The underlying data had never been subject to any court scrutiny,"
he says. "Defence experts were trusting that the scientists had interpreted the
data correctly. This perpetuated the myth that DNA is infallible."</P>
<P>Peter Gill of the University of Strathclyde in Glasgow, UK, a former analyst
at the UK's Forensic Science Service, admits that there is a problem. "There's a
considerable lack of understanding, not just from the public, but from the
judges and lawyers."</P>
<P>The problem is not confined to the UK. "In our experience, examination of the
underlying data frequently reveals limitations or problems that would not be
apparent from the lab report," says William Thompson of the University of
California, Berkeley, who acts as an expert witness on DNA. However, "forensic
DNA analysts tell us that they receive requests [for DNA lab reports] from
defence lawyers in only 10 to 15 per cent of cases in which their tests
incriminate a suspect," Thompson says.</P>
<P>Even when the defence makes a legitimate challenge, the public rarely hears
about it. In the UK, defence lawyers are granted access to DNA data on the
condition that they only use it in the case in question. If the questioned
evidence is dropped before it gets to court, this never becomes public, says
Allan Jamieson of the <A href="http://www.theforensicinstitute.com/"
target=nsarticle>Forensic Institute</A> in Glasgow.</P></DIV>
<DIV id=hldartIssueInfo></DIV></DIV></DIV></DIV></FONT></DIV></BODY></HTML>