Decline effect

The decline effect may occur when scientific claims receive decreasing support over time. The term was first described by parapsychologist Joseph Banks Rhine in the 1930s to describe the disappearing of extrasensory perception (ESP) of psychic experiments conducted by Rhine over the course of study or time. The term was once again used in a 2010 article by Jonah Lehrer published in The New Yorker.^[1]

Examples from Lehrer's article

In his article, Lehrer gives several examples where the decline effect is allegedly showing. In the first example, the development of second generation anti-psychotic drugs, reveals that the first tests had demonstrated a dramatic decrease in the subjects' psychiatric symptoms.^[1] However, after repeating tests this effect declined and in the end it was not possible to document that these drugs had any better effect than the first generation anti-psychotics.

Experiments done by Jonathan Schooler was trying to prove that people describing their memories were less able to remember them than people not describing their memories. His first experiments were positive, proving his theory about verbal overshadowing, but repeated studies showed a significant declining effect.^[1]

In 1991, Danish zoologist Anders Møller discovered a connection between symmetry and sexual preference of females in nature. This sparked a huge interest in the topic and a lot of follow-up research was published. In three years following the original discovery, 90% of studies confirmed Møller's hypothesis. However, the same outcome was published in just four out of eight research papers in 1995, and only a third in next three years.^[2]

Explanations

One of the explanations of the effect can regression toward the mean. This is a statistical phenomenon happening when a variable is extreme on the first experiments and by later experiments tend to regress towards average.

Another reason may be the publication bias: scientists and scientific journals prefer to publish positive results of experiments and tests over null results, especially with new ideas.^[1] As a result, the journals may refuse to publish papers that do not prove that the idea works. Later, when an idea is accepted, journals may refuse to publish papers that support it.

In the debate that followed the original article, Lehrer answered some of the questions by claiming that scientific observations might be shaped by one's expectations and desires, sometimes even unconsciously, thus creating a bias towards the desired outcome.^[2]

A significant factor contributing to the decline effect can also be the sample size of the scientific research, since smaller sample size is very likely to give more extreme results, suggesting a significant breakthrough, but also a higher probability of an error. Typical examples of this effect are the opinion polls, where those including a larger number of people are closer to reality than those with a small pool of respondents.^[3]

Contesting views

Several actors have contested Jonah Lehrer's view of the decline effect being a problematic side of the phenomenon, as presented in his New Yorker article. "The decline effect is troubling because it reminds us how difficult it is to prove anything. We like to pretend that our experiments define the truth for us. But that's often not the case. Just because an idea is true doesn't mean it can be proved. And just because an idea can be proved doesn't mean it's true. When the experiments are done, we still have to choose what to believe." ^[1]

Steven Novella also challenges the Lehrer's view of the decline effect, arguing that Lehrer is concentrating on new discoveries on the cutting edge of scientific research and applying the conclusions to all areas of science. Novella points out that most of the examples used by Lehrer come from medicine, psychology and ecology, scientific fields most influenced by a complex human aspect and that there is not much evidence of the decline effect in other areas of science, such as physics.^[4]

Another scientist, Paul Zachary Myers, is also contesting Lehrer's view on the decline effect being a surprising phenomenon in science, claiming that: "This isn't surprising at all. It's what we expect, and there are many very good reasons for the shift." ^[5]

Lehrer's claims about difficulty of proving anything and publication bias find support from Jerry A. Coyne. Coyne claims that in the fields of genetics and evolutionary biology, almost no research is replicated and there is a premium motivation offered for publishing positive results of research studies. However, he also contests Lehrer's approach of applying conclusions on all fields of science, stating that in physics, chemistry or molecular biology, previous results are constantly repeated by others in order to progress in their own research.^[6]

Work of John Ioannidis offers some explanation of the decline effect itself in science. Ioannidis states that early research is usually small and more prone to highly positive results supporting the original idea, including early confirmatory studies. Later, as larger studies are being made, they often show regression to the mean and a failure to repeat the early exaggerated results.^[7]

Criticism

One concern that some ^[8] have expressed is that Lehrer's article may further fuel people's skepticism to academic science. It was long believed that Lehrer's article originally hinted that academic science is not as rigid as people would like to believe. It is especially the article's ending that has upset many scientists and led to broad criticism of the article. Lehrer ends the article by saying: "Just because an idea is true doesn't mean it can be proved. And just because an idea can be proved doesn't mean it's true. When the experiments are done, we still have to choose what to believe." This has upset scientists in the scientific community. Many have written back to Lehrer and questioned his agenda. Some have characterized Lehrer's assertion as "absurd", while others claiming that Lehrer is trying to use publication bias as an excuse for not believing in anything.^[8]

As an answer to the many comments Lehrer received upon publishing the article, Lehrer published a comment on his blog, The Frontal Cortex,^[2] where he denied that he was implicitly questioning science and scientific methods in any way. In the same blog comment, Lehrer stated that he was not questioning fundamental scientific theories such as the theory of evolution by natural selection and global warming by calling them "two of the most robust and widely tested theories of modern science".

A further clarification was published as a follow-up note in The New Yorker.^[2] In this note, entitled "More Thoughts on the Decline Effect", Lehrer tries mainly to answer the critics by giving examples where scientific research has both failed and succeeded. As an example, Lehrer uses Richard Feynman's commencement speech at Caltech University in 1974 as a starting point. In his commencement speech, Feynman used Robert Millikan's and Harvey's oildrop experiment to measure the charge of an electron as an example to illustrate how selective reporting can bias scientific results. On the other hand, Feynman finds solace in the fact that other scientists will repeat other scientists experiments and hence, the truth will win out in the end.

Lehrer once again uses the follow-up note to deny that his original intention was to support people denying well verified scientific theories such as the natural selection and climate change. Instead, he wishes that: "we'd spend more time considering the value of second-generation antipsychotics or the verity of the latest gene-association study". In the other parts of the follow-up note, Lehrer briefly discusses some of the creative feedback he has received in order to reduce publication bias. Lehrer does not give his support explicitly to any specific idea. The follow-up article ends with Lehrer once again claiming that the decline effect is a problem in today's science, but that science will eventually find a tool to deal with the problem.

See also

• Cargo cult science

References

1. ^ ^{a b c d e} Jonah_Lehrer (2010). "The Thruth Wears Off". The New Yorker. http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer. 
2. ^ ^{a b c d} Jonah Lehrer (2010). "The Mysterious Decline Effect". Wired. http://www.wired.com/wiredscience/2010/12/the-mysterious-decline-effect. 
3. ^ John Allen Paulos (2010). "The decline effect and why scientific 'Truth' so often turns out wrong". ABC News. http://abcnews.go.com/Technology/WhosCounting/decline-effect-scientific-truth-turns-wrong/story?id=12510202. 
4. ^ Steven Novella (2010). "The Decline Effect". Neurologicablog. http://theness.com/neurologicablog/?p=2580/. 
5. ^ Paul Zachary Myers (2010). "Science is not dead". Scienceblogs. http://scienceblogs.com/pharyngula/2010/12/science_is_not_dead.php/. 
6. ^ Jerry Allen Coyne (2010). "Why Evolution is True". Wordpress. http://whyevolutionistrue.wordpress.com/2010/12/10/the-decline-effect-can-we-demonstrate-anything-in-science/. 
7. ^ David Gorski (2010). "The "decline effect": Is it a real decline or just science correcting itself?". Science-based Medicine. http://www.sciencebasedmedicine.org/?p=8987/. 
8. ^ ^{a b} John Horgan (2010). "The truth we'll doubt: Does the decline effect mean that all science is "truthy"?". Scientific America. http://www.scientificamerican.com/blog/post.cfm?id=the-truth-well-doubt-does-the-decli-2010-12-13.