<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=iso-8859-1" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 8.00.6001.18939">
<STYLE></STYLE>
</HEAD>
<BODY bgColor=#ffffff>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2>From <EM>New Scientist</EM>, August 14 - 20, 2020</FONT></DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2><A
href="http://www.newscientist.com/">http://www.newscientist.com/</A></FONT></DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2>
<H1>Time to rethink DNA evidence </H1></FONT></DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV>
<P class=infuse>THEY call it the CSI effect. The seemingly endless succession of
glossy TV shows about forensic science has created a popular perception that the
dazzling array of techniques now available can produce fast and firm evidence
for convictions.</P>
<P class=infuse>It is well known that traditional methods such as bite-mark and
fingerprint analysis rely to some extent on analysts' opinions, and as such they
are viewed with a healthy measure of scepticism. However, DNA evidence has so
far escaped this treatment.</P>
<P class=infuse>Profiling is generally seen as routine, infallible and always
able to get its man. One recent study found that jurors rated DNA evidence as 95
per cent accurate, while mock trials showed that they were more likely to
convict when presented with DNA evidence than other types of evidence.</P>
<DIV class="quotebx bxbg">
<DIV class=quoteopen>
<DIV class=quoteclose>
<DIV class="quotebody lowlight"><QUOTE><QUOTETEXT>Jurors see DNA evidence as
highly accurate and are more likely to convict when it is
presented</QUOTETEXT></QUOTE> </DIV></DIV></DIV></DIV>
<P class=infuse>But, as we demonstrate <A
href="http://www.newscientist.com/article/mg20727733.500-fallible-dna-evidence-can-mean-prison-or-freedom.html">in
this issue</A>, DNA profiling is far from perfect, and in many cases can run
into the same problems of subjectivity and bias seen with the more old-fashioned
techniques.</P>
<P class=infuse>This is an evolving problem, one rarely encountered in the early
days, when analysts only tended to tackle large samples of DNA from one or two
people. Forensic scientists are now increasingly swabbing crime scenes for DNA
evidence as a matter of routine, which means samples are often very small or
contain DNA from more than one person. These factors make analysis far more
challenging.</P>
<P class=infuse>Even though doubts were raised about DNA evidence as long ago as
the early 1990s, <A
href="http://www.newscientist.com/article/mg12717262.300-doubts-raised-over-accuracy-of-dna-fingerprinting.html">albeit
for quite</A> <A
href="http://www.newscientist.com/article/mg13418171.700-courtroom-battle-over-genetic-fingerprinting.html">different</A>
<A
href="http://www.newscientist.com/article/mg13718632.700-dna-fingerprinting-back-in-the-dock-the-case-of-a-man-cleared-of-armed-robbery-has-badly-dented-the-credibility-of-genetic-fingerprinting-in-british-courts.html">reasons</A>,
it is still seen as rock solid. There is, of course, plenty of very convincing
DNA evidence, but little distinction is made, either in labs or in court,
between samples of different quality: DNA is just DNA.</P>
<P class=infuse>There's a deeper problem, too: a pervasive idea that the role of
forensic science is to convict criminals, rather than to evaluate the available
evidence and present it in the right context.</P>
<P class=infuse>One way to begin to tackle these issues would be to use
statistical methods to ensure that the correct weight is placed on each piece of
evidence and allow juries to better assess the likelihood of a suspect's
innocence or guilt. In next week's issue, we will examine the options on offer
and show that it is possible to use them to remove some of the subjectivity
around DNA analysis and highlight those worrying cases where the evidence is
feeble.</P>
<P class=infuse>However, few labs currently use the recommended statistical
analysis, despite the fact that there are several methods available. Why is
this? Many scientists we spoke to complained about a lack of funding. Peter
Gill, former principal research scientist for the UK's Forensic Science Service,
points out that even though improved statistical methods were devised over a
decade ago, the lack of investment means that they have not yet been translated
into practical techniques. If and when they are, the resulting software packages
should be open-source, so that all labs can do the same analysis. For there to
be real justice, labs must have equal access to all the latest technology.</P>
<P class=infuse>If forensic science is ever to live up to the hype of the CSI
effect, statistics need to become as cool and important as the techniques
deployed at the crime scene.</P></DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2></FONT> </DIV>
<DIV><FONT size=2>______________________________________</FONT></DIV>
<DIV><FONT size=2>
<H1>Fallible DNA evidence can mean prison or freedom</H1></FONT></DIV>
<DIV><FONT size=2>
<P class=infuse>YOU are the juror: would you trust DNA evidence? Most people
regard it as near infallible- it produces the right result or no result,
exonerating the innocent and securing convictions where other evidence
fails.</P>
<P class=infuse>But DNA is not as objective as you might think. In the first of
a two-part investigation, <I>New Scientist</I> reveals that much of the DNA
analysis now conducted in crime labs can suffer from worrying subjectivity and
bias. We asked forensic analysts to interpret a sample of real DNA evidence and
found that they reached opposing conclusions about whether the suspect matched
it or not. Our subsequent survey of labs around the world also shows that there
are significant inconsistencies in the guidelines on how to interpret a sample.
The findings suggest that the difference between prison and freedom could often
rest on the opinions of a single individual.</P>
<DIV class="quotebx bxbg">
<DIV class=quoteopen>
<DIV class=quoteclose>
<DIV class="quotebody lowlight"><QUOTE><QUOTETEXT>Much of the DNA analysis
conducted can suffer from worrying subjectivity and bias</QUOTETEXT></QUOTE>
</DIV></DIV></DIV></DIV>
<P class=infuse>The introduction of DNA evidence to the courtroom in the mid
1980s revolutionised forensic science, resulting in thousands of convictions and
exonerating 255 wrongly convicted people so far in the US alone. The reason for
more than 50 per cent of these <A
href="http://www.innocenceproject.org/understand/Unreliable-Limited-Science.php"
target=nsarticle>wrongful convictions</A> was unvalidated or improper forensic
testing, including incorrect hair, blood or <A
href="http://www.newscientist.com/article/mg18725174.500-how-far-should-fingerprints-be-trusted.html">fingerprint
analysis</A>.</P>
<P class=infuse>"It's not unreasonable to hold up DNA as a way that the rest of
forensic science should be done," says <A
href="http://socialecology.uci.edu/faculty/wcthomps" target=nsarticle>William
Thompson</A> of the University of California, Irvine, and an occasional expert
witness on DNA. "It is better validated, and often more carefully done and more
rigorously interpreted than many areas of forensic science."</P>
<P class=infuse>That's not the same as saying DNA is perfect, however. In a
growing number of cases, <A
href="http://www.newscientist.com/article/dn8579-dna-profiling-moves-to-the-scene-of-the-crime.html">DNA
samples taken from crime scenes</A> produce partial profiles, partly because
smaller samples are collected. "Labs are trying to get more samples and they're
trying to [get results from] lower and lower amounts of DNA," says <A
href="http://www.cstl.nist.gov/strbase/butler.htm" target=nsarticle>John
Butler</A>, head of the US National Institute of Standards and Technology's
genetics group, which aims to improve standards in DNA testing.</P>
<P class=infuse>A standard DNA profile consists of a series of peaks relating to
the number of repeating stretches of DNA found in certain genetic sequences, or
alleles (<A
href="http://www.newscientist.com/articleimages/mg20727733.500/0-fallible-dna-evidence-can-mean-prison-or-freedom.html">see
diagram</A>). The repeats occur at specific locations on the chromosomes, called
loci, and there are two alleles at each locus- one inherited from each parent.
The number of repeats in each allele varies widely between individuals, allowing
a person to be identified this way. Labs in the US typically look at 13 loci,
while UK labs tend to look at 10.</P>
<P class=infuse>Yet in partial profiles, alleles may fail to show up, a
phenomenon called "drop-out". False peaks in the profile created by
imperfections in the analysis machine may also be mistaken for alleles. This is
called "drop-in".</P>
<P class=infuse>It gets more complicated <A
href="http://www.newscientist.com/article/mg18725163.800-television-shows-scramble-forensic-evidence.html">when
several people's DNA is mixed</A> <A
href="http://www.newscientist.com/article/mg20727733.500-fallible-dna-evidence-can-mean-prison-or-freedom.html?full=true#bx277335B1">(see
"Mix and mismatch").</A> Butler has reviewed more than 5000 DNA samples from 14
US labs and found that mixing is a common occurrence: 34 per cent of the samples
he studied included DNA from two people, while 11 per cent were three or
four-person mixtures.</P>
<P class=infuse>Interpreting alleles in a mixed or partial sample is where the
subjective opinion of an analyst could play a part. To test this, <I>New
Scientist</I> teamed up with <A
href="http://www.icn.ucl.ac.uk/Staff-Lists/MemberDetails.php?Title=Dr&FirstName=Itiel&LastName=Dror"
target=nsarticle>Itiel Dror</A>, a neuroscientist at University College London
and head of <A href="http://www.cci-hq.com/" target=nsarticle>Cognitive
Consultants International</A>, and <A
href="http://boisestate.edu/biology/hampikianoldweb/hampikian.shtml"
target=nsarticle>Greg Hampikian</A> of Boise State University in Idaho.</P>
<P class=infuse>We took a mixed sample of DNA evidence from an actual crime
scene- a gang rape committed in Georgia, US- which helped to convict a man
called Kerry Robinson, who is currently in prison. We presented it, and
Robinson's DNA profile, to 17 experienced analysts working in the same
accredited government lab in the US, without any contextual information that
might bias their judgement.</P>
<P class=infuse>In the original case, two analysts from the Georgia Bureau of
Investigation concluded that Robinson "could not be excluded" from the crime
scene sample, based on his DNA profile. (A second man convicted of the same
crime also testified that Robinson was an assailant, in return for a lesser jail
term.) Each of our 17 analysts independently examined the profiles from the DNA
mixture, the victim's profile and those of two other suspects and was asked to
judge whether the suspects' profiles could be "excluded", "cannot be excluded"
or whether the results were "inconclusive".</P>
<P class=infuse>If DNA analysis were totally objective, then all 17 analysts
should reach the same conclusion. However, we found that just one agreed with
the original judgement that Robinson "cannot be excluded". Four analysts said
the evidence was inconclusive and 12 said he could be excluded.</P>
<P class=infuse>"Fingerprinting and other forensic disciplines have now accepted
that subjectivity and context may affect their judgement and decisions," says
Dror. "It is now time that DNA analysts accept that under certain conditions,
subjectivity and even bias may affect their work." Dror presented the results at
the Green Mountain DNA conference in Burlington, Vermont, last month.</P>
<DIV class="quotebx bxbg">
<DIV class=quoteopen>
<DIV class=quoteclose>
<DIV class="quotebody lowlight"><QUOTE><QUOTETEXT>Varying the identity of the
suspect changed the answers that DNA analysts gave</QUOTETEXT></QUOTE>
</DIV></DIV></DIV></DIV>
<P class=infuse><A href="http://www.christinefunk.net/page2.html"
target=nsarticle>Christine Funk</A>, an attorney in the Office of the Public
Defender for the State of Minnesota, says the results of <I>New Scientist</I>'s
survey have profound implications for criminal justice. "The difference between
prison and freedom rests in the hands of the scientist assigned the case," she
says.</P>
<P class=infuse>Eric Buel, director of <A
href="http://www.dps.state.vt.us/cjs/vfl/" target=nsarticle>Vermont Forensic
Laboratory</A> in Waterbury agrees that there is a problem, although he doesn't
think it will apply to every lab. "I would be a little bit concerned if one
person excludes, and one person includes him. At the end of the day, we all
should come to about the same answer on this stuff." Both he and Butler suggest
that inconsistencies in analysts' training may be partly to blame.</P>
<H3 class=crosshead>Variable thresholds</H3>
<P class=infuse>The problem of subjective interpretation could be further
exacerbated by differences in procedure between labs. According to a second
survey conducted by <I>New Scientist</I>, many crime labs set their own
thresholds for how high a peak must be to demonstrate the presence of an allele,
and these can be inconsistently applied.</P>
<P class=infuse><I>New Scientist</I> sent a questionnaire to crime labs in the
US, Canada, UK and Australia. Of the 19 that replied, we found that four labs
routinely allow analysts to use their discretion when interpreting peaks whose
height is below their statistical cut-off. A further two said that although it
wasn't routine, there were circumstances when analysts could use their
discretion. Fifteen labs said that they did not have a minimum requirement below
which someone would be excluded from a mixture.</P>
<P class=infuse>So what can be done? This year, the <A
href="http://www.nfstc.org/pdi/Subject10/pdi_s10_m03_01_d.htm"
target=nsarticle></A>Scientific Working Group on DNA Analysis Methods (SWGDAM),
which issues guidance to US labs performing forensic DNA analysis, <A
href="http://www.cstl.nist.gov/strbase/pub_pres/NIJ2010-SWGDAM-Guidelines.pdf"
target=nsarticle>published new recommendations regarding the interpretation of
forensic DNA</A>. These include a suggestion that labs develop strict criteria
for deciding what denotes the presence of an allele, and what amount of DNA
constitutes the minimum for a profile to be constructed. Labs should also
document and define any assumptions used in the analysis of a mixture. "The
bottom line is that you want to be as consistent and accurate as possible," says
Butler, who chaired the SWGDAM committee.</P>
<P class=infuse>It seems lab managers would welcome consistent rules. Forensic
lab directors at the 19 labs we surveyed also provided their views about how
their analysis is currently done: 15 either agreed or strongly agreed that
interpretation procedures should be based on national standards, and 11 agreed
or strongly agreed that decisions over alleles should not be based on analyst
opinion.</P>
<P class=infuse>Labs must also take steps to avoid bias. Butler says that some
labs continue to insist upon seeing suspect profiles before analysing evidence
from the crime scene, which could lead to biased decision-making <A
href="http://www.newscientist.com/article/mg20727733.500-fallible-dna-evidence-can-mean-prison-or-freedom.html?full=true#bx277335B2">(see
"Crime Scene Investigation: Impartiality").</A> Analysts also often know too
much about a suspect and other evidence to be impartial, and public labs often
have close ties to police. "Crime labs, including DNA labs, should not be under
the control of a law enforcement agency," says one US analyst, who wished to
remain anonymous. "We are scientists, not cops or prosecutors."</P>
<P class=infuse>In our survey using the Georgia sample, respondents were blinded
to contextual information about the case. <A
href="http://darwin.bio.uci.edu/~mueller/" target=nsarticle>Larry Mueller</A> of
the University of California in Irvine says we may have seen different results
if the data had been presented to them by police officers or prosecution
lawyers. "The difference between you giving them the data and saying 'what do
you make of it?' and the local district attorney giving them the data and
saying: 'We've arrested someone, is his profile in here?' is huge," he says.</P>
<P class=infuse>Bruce Budowle, a former head of the FBI's DNA lab, would also
like to see labs employ a second analyst to review initial conclusions, and all
of this data be made available to defence teams.</P>
<P class=infuse>Eighteen of the labs that we surveyed said they already conduct
independent reviews. However, in the majority of cases, the reviewer is allowed
to see the first analyst's conclusions, as well as the original data. "Technical
peer reviews are a good step, but I can point to several examples where
peer-reviewers just rubber-stamp the cases," says a different US analyst, who
works in a private DNA lab that carries out case work for the police. In the
case of a disagreement, 15 of the labs said a supervisor would be called in to
make the final call, but only two labs said that this disagreement would be
documented in their final report.</P>
<P class=infuse>Still, when done correctly, DNA remains a powerful tool for
fighting crime, says Budowle. "Are there cases done wrongly? Absolutely, there
have been cases," he says. "Are they the vast majority? No, it's a small
number."</P>
<P class=infuse>Buel adds that some DNA labs are beginning to accept that
improvements need to be made. "DNA analysis is still relatively young. It may
take us a while to make sure we're all playing by the same set of rules," he
says. "At the end of the day, I think we're all interested in making sure we get
the right person. DNA is headed in the right direction, but as in any field, you
can make it better."</P>
<DIV class="artbx bxbg">
<H3 id=bx277335B1>Mix and mismatch</H3>
<P>Mixed-up DNA from crime scenes already causes headaches for analysts; now it
seems it can even be difficult to tell how many people's DNA is present in a
sample.</P>
<P>Dan Krane of Wright State University in Dayton, Ohio, and his team took 959
full DNA profiles and modelled all the possible three and four-person mixtures
that could arise from them. They found that 3 per cent of three-person mixtures
could be mistaken for those of two people, and more than 70 per cent of
four-person mixtures could be mistaken for two or three-person mixtures
(<I>Journal of Forensic Sciences</I>, <A
href="http://dx.doi.org/10.1520/JFS2004475" target=nsarticle>DOI:
10.1520/JFS2004475</A>).</P>
<P>"If you can't determine how many contributors there were, it is ludicrous to
suggest that you can tease apart who those contributors were or what their DNA
profiles were," says Krane.</P></DIV>
<DIV class="artbx bxbg">
<H3 id=bx277335B2>Crime scene investigation: Impartiality</H3>
<P>Sarah Beazley woke up with a hangover and her clothes in disarray. She had
attended a party the night before, where she admitted to getting high on drugs
and alcohol. Although she couldn't remember the details, she suspected that she
may have been sexually assaulted. When Sarah (not her real name) went to the
police, they found traces of saliva on her breasts.</P>
<P>Sarah's story- a real case in California- provides a good example of how DNA
analysts often use subjective judgements to interpret evidence, depending on the
case information they have been told. As in many cases, the DNA profile from the
saliva sample (shown below) was incomplete: the identifying peaks could only be
obtained for three sites, or loci, on the DNA. On a complete sample, US labs
would look at 13 loci. The number of repeating DNA sequences at each of these
loci varies between people, and so analysts can eliminate or implicate suspects
if their peak profiles line up with those in samples taken from a crime
scene.</P>
<P>In Sarah's case, there were four possible suspects: here given the pseudonyms
Tom, Dick, Harry and Sally. Each provided DNA swabs, and their number of repeats
at each relevant locus was recorded.</P>
<P>To test how analysts would interpret these samples, William Thompson of the
University of California in Irvine and <A
href="http://www.wright.edu/biology/department/directory/faculty/krane/"
target=nsarticle>Dan Krane</A> of Wright State University in Dayton, Ohio, who
have both acted as DNA expert witnesses, presented the information to around
1000 analysts at various forensics meetings. Thomson would vary the identity of
the suspect and contextual information about the case. He found this produced a
variety of answers (<I>Law, Probability and Risk</I>, <A
href="http://dx.doi.org/10.1093/lpr/mgp013" target=nsarticle>DOI:
10.1093/lpr/mgp013</A>).</P>
<P>In the first instance, analysts were showed only Tom's sample, and told that
he was the suspect. Thomson also said that there were concerns that Tom's
profile couldn't account for all of the peaks seen in the saliva sample: his
profile doesn't quite match up at locus 1 or locus 3. "At that point, several
analysts interrupted to say that I did not know what I was talking about," says
Thompson. The interruptors explained that the non-matching peaks were just
random fluctuations caused by, say, a flaw in the analysis machine. Conclusion:
it could be Tom's saliva.</P>
<P>However, when a second group of analysts were presented with just Dick's
profile and told that he was the suspect, they concluded that the "flaws"
dismissed by the first group were true peaks. So Dick could be a match too.</P>
<P>Thompson then presented the case to a third group of analysts, this time
showing only Harry as the suspect. He heard a similarly subjective story. Harry
didn't match at locus 1, but some information could have been lost from the
saliva sample, the analysts said. And some of the repeats at locus 3 must be
hidden by an artefact or other DNA sample in the saliva.</P>
<P>Finally, Krane presented Sally to a fourth group. Even though one of her
alleles at both locus 2 and locus 3 did not match, analysts insisted that she
could not be excluded, in part because the evidence sample could be a mixture of
more than one person's DNA.</P>
<P>Although these informal experiments do not necessarily reflect how labs
interpret evidence on a day-to-day basis, "about a third of the cases that I'm
asked to review have elements of the Tom, Dick, Harry and Sally story," says
Krane.</P>
<P>So who was convicted in the real case? The answer is Tom.</P></DIV>
<P><I>Additional research by Charlotte King and Caitlin Stier</I></P>
<DIV id=hldartIssueInfo></DIV></FONT></DIV>
<DIV><FONT size=2></FONT> </DIV><FONT size=2>
<DIV><BR>Wayne A. Fox<BR>1009 Karen Lane<BR>PO Box 9421<BR>Moscow, ID
83843</DIV>
<DIV> </DIV>
<DIV><A href="mailto:waf@moscow.com">waf@moscow.com</A><BR>208
882-7975<BR></FONT></DIV></BODY></HTML>