Document information

2 votes

Reducing the Probability of False Positive Research Findings by Pre-Publication Validation – Experience with a Large Multiple Sclerosis Database

Martin M. Daumer¹, Ulrike U. Held¹, Katja K. Ickstadt², Moritz M. Heinz³, Siegfried S. Schach², & George G. Ebers⁴

Correspondence: (Login to view email address)

1. Sylvia Lawry Centre for MS Research, Hohenlindener Str. 1, 81677 Munich, Germany
2. Department of Statistics, University of Dortmund, 44221 Dortmund, Germany
3. Institute for Medical Biometry, Epidemiology and Computer Science, Clinic of Gutenberg’s University Mainz, Germany
4. University Dept of Clinical Neurology, Oxford University, UK

DOC (105 KB) PDF (92.3 KB)

Document Type:

Manuscript

Date:

Received 12 July 2007 06:25 UTC; Posted 12 July 2007

Subjects:

Neuroscience, Bioinformatics

Tags:

• Validation testing
• Multiple Sclerosis

Abstract:

Objective
We have assessed the utility of a pre-publication validation policy in reducing the probability of publishing false positive research findings.
Study design and setting
The large database of the Sylvia Lawry Centre for Multiple Sclerosis Research was split in two parts: one for hypothesis generation and a validation part for confirmation of selected results. We present case studies from 5 finalized projects that have used the validation policy and results from a simulation study.
Results
In one project, the “relapse and disability” project as described in section II (example 3), findings could not be confirmed in the validation part of the database. The simulation study showed that the percentage of false positive findings can exceed 20% depending on variable selection.
Conclusion
We conclude that the validation policy has prevented the publication of at least one research finding that could not be validated in an independent data set (and probably would have been a “true” false-positive finding) over the past three years, and has led to improved data analysis, statistical programming, and selection of hypotheses. The advantages outweigh the lost statistical power inherent in the process.

Discussion

Votes:

2 votes

(Login to vote)

Comments:

0 comments

(Login to post a comment)

Share:

(Login to share with a colleague)

Additional information

License:

This document is licensed to the public under the Creative Commons Attribution 2.5 License

How to cite this document:

Daumer, Martin, Held, Ulrike, Ickstadt, Katja, Heinz, Moritz, Schach, Siegfried, and Ebers, George. Reducing the Probability of False Positive Research Findings by Pre-Publication Validation – Experience with a Large Multiple Sclerosis Database. Available from Nature Precedings <http://hdl.handle.net/10101/npre.2007.433.1> (2007)

Version info:

Other versions of this document in Nature Precedings

None.

Other versions of this document elsewhere on the web

None known.

Submit a new version link