[Vision2020] Science is Self Correcting: Lessons from the Recent Arsenic Based Life Controversy

Ted Moffett starbliss at gmail.com
Fri Dec 31 14:16:04 PST 2010


http://www.realclimate.org/index.php/archives/2010/12/science-is-self-correcting-lessons-from-the-arsenic-controversy/#more-5845

Science is self-correcting: Lessons from the arsenic controversy
Filed under: Scientific practice— group @ 29 December 2010

Recent attention to NASA’s announcement of ‘arsenic-based life’ has
provided a very public window into how science and scientists operate.
Debate surrounds the announcement of any controversial scientific
finding. In the case of arseno-DNA, the discussion that is playing out
on the blogs is very similar to the process that usually plays out in
conferences and seminars. This discussion is a core process by which
science works.

The arseno-DNA episode has displayed this process in full public view.
If anything, this incident has demonstrated the credibility of
scientists, and should promote public confidence in the scientific
establishment.

The story begins with a long-standing scientific consensus backed by
an enormous amount of data: DNA is made with a phosphate backbone.
Alternative backbones, such as arsenate, have long been considered
unlikely for theoretical reasons.

Nonetheless, despite this consensus, reputable scientists have
promoted the study of alternatives challenging the prevailing view.
And NASA has willingly funded these studies.

Lesson one: Major funding agencies willingly back studies challenging
scientific consensus.

The research team, Felisa Wolfe-Simon and colleagues, behind this
study collected data and concluded that they had sufficient evidence
to demonstrate incorporation of arsenate into bacterial DNA. Although
the data were preliminary in nature, Science accepted the manuscript
(pdf). With a high profile, potentially groundbreaking paper about to
be published, NASA announced a press conference to publicize the
findings.

Lesson two: Most everyone would be thrilled to overturn the consensus.
Doing so successfully can be a career-making result. Journals such as
Science and Nature are more than willing to publish results that
overturn scientific consensus, even if data are preliminary – and
funding agencies are willing to promote these results.

Within days of the arsenic paper’s publication, strong criticism of
the study began to appear on scientific blogs. These blogs attracted
the attention of the mainstream scientific press. Soon thereafter,
media reported the wide skepticism within the scientific community –
with some scientists going so far as to say that the paper should not
have been published.

These scientific criticisms opened the door to those wishing to
discredit science and the peer-review process, with the contrarian
blogs suggesting that this study demonstrates that peer-review is
“broken”. A comment on Watts’ blog summarizes their thinking:

It’s amazing how fast the scientific community came out to attack NASA
for what they claim is plainly flawed science. Then again, NASA isn’t
funding any of the attackers.

In the Climategate mess however, we still have heard very little from
an awful lot of so-called scientists who should have been saying a lot
more about flawed science but are too afraid to lose their grant
money.

This raises an interesting question: just who is critiquing the NASA
study? It turns out that many of the critics are also NASA-funded. In
fact, many prominent critics of this study are funded by the NASA
Astrobiology Institute – the very same program that funded the arsenic
study.

Carl Zimmer gives us several examples:

Norm Pace offers the critique: “Low levels of phosphate in growth
media, naive investigators and bad reviewers are the stories here”.
Shelley Copley suggested, “this paper should not have been published”
Roger Summons remarked that a critical experiment was left undone, and
backed the critical blog analysis of his NASA-funded former student.
Michael Russell agreed with blogosphere critics, and offered his own
critique of the study based on cosmic ratios of phosphorus to arsenic.
Russell is a member of the Astrobiology Institute, as well as an
employee of NASA’s Jet Propulsion Laboratory.
Forest Rohwer observed, “the experimental evidence in the paper is
pretty weak.”
George Cody says he “cannot accept this claim until such an experiment
[mass spectrometry] (easily done) is performed.”
Steven Benner was an early skeptic. To NASA’s credit, they invited him
to present his criticisms at the press conference. He has said “we are
not expecting this result to survive”.

Each of these scientists is affiliated with NASA Astrobiology.

Lesson three: Scientists offer opinions based on their scientific
knowledge and a critical interpretation of data. Scientists willingly
critique what they think might be flawed or unsubstantiated science,
because their credibility – not their funding – is on the line.

Regardless of whether or not ‘arseno-DNA’ survives the test of time
and further study, scientists have shown that they will rigorously
criticize science perceived as flawed, with no fear of reprisal from
funding agencies.

This is the key lesson to take from this incident, and it applies to
all scientific disciplines: peer-review continues after publication.
Challenges to consensus are seriously entertained – and are accepted
when supported by rigorous data. Poorly substantiated studies may
inspire further study, but will be scientifically criticized without
concern for funding opportunities. Scientists are not “afraid to lose
their grant money”.

Finally, there is the issue of how scientists who publish papers that
generate credible blog reactions should in turn react. In times past,
it was simple to wait for properly crafted letters and comments to be
sent in to the journal. This gave fixed targets to deal with and
allowed for considered reflection and response; discussions would
perhaps be published 6 months to year later. But today, serious
criticisms can arrive immediately (as seen above). Nature (perhaps
with a little schadenfreude) had an op-ed suggesting that the authors
on this (Science) paper should be more strongly engaged in the
reaction, while Science had a plea from the lead author for a little
patience, since they were clearly a little overwhelmed.

In our view, this needs to be thought about clearly on a case by case
basis. Some criticisms (that for instance accuse the authors of
deliberate fraud or misconduct based on a dislike of the conclusions)
are not worth rapidly responding to, but it is worth trying to head
off any misinterpretations that might be emerging. Short form papers
(even with copious supplementary information) do not provide full
context for the results in themselves, and so putting together a
response to frequently asked questions is certainly useful (as Dr.
Wolfe-Simon and colleagues have). This doesn’t replace the need for
technical commentary to pass via the peer-review process though. In
the end, that is what people will refer back to.

------------------------------------------
Vision2020 Post: Ted Moffett



More information about the Vision2020 mailing list