Bias erodes translational value in animal spinal cord injury studies

An analysis led by researchers from The Ohio State University College of Medicine at The Ohio State University Wexner Medical Center found that bias in the design and reporting of spinal cord injury (SCI) studies in animals results in overestimation of the effectiveness of a potential treatment to prevent infections in human patients with SCI.

The research findings are published online in the journal Neurology.

The study identified three major findings:

• Evidence of widespread bias in animal SCI experiments which have set out to test new interventions in order to improve neurological recovery.
• Factoring in the impact of bias unmasks considerable inflated effect sizes mirroring overestimated efficacy.
• Overestimated efficacy of an intervention tested in animals will results in an underpowered clinical trial unable to detect a potential real but much smaller effect.

"Despite tremendous progress being made in the understanding of underlying mechanisms after spinal cord injury, the subsequent bench-to-bedside translation remains a very fragile process. Improving the predictive value of preclinical SCI research is a requirement to improve the chances for translational success. Inflated, too optimistic effect sizes in animal studies will provide false leads for subsequent clinical testing," said principal investigator Dr. Jan M. Schwab, neurologist and physician at Ohio State's Neurological Institute.

Schwab collaborated with researchers from several institutes in Australia, Germany and the United Kingdom as part of the international CAMARADES consortium (Collaborative approach to Meta-Analysis and Review of Animal Data from Experimental Studies).

Researchers analyzed data from 549 preclinical published SCI studies comprising data from more than 9,500 animals. Only less than half of the studies indicated blinding of the experiments. Lack of blinding experiments can lead to evaluation bias, and is associated with an overstated effect size of 7.2% (locomotor recovery) after preclinical SCI testing, said Schwab, who is also medical director of the Belford Center for Spinal Cord Injury at Ohio State.

Another source of bias occurs when negative or non-confirmatory results of experiment are not published, leading to an overestimation of efficacy, researchers found. Regression models unmasked that depending on the type of intervention, up to 41% of SCI experiments remain unpublished. Inclusion of these theoretical missing studies suggested an overestimation of efficacy by up to 14%.

"Clinical trials are costly and resource intense. Improving the predictive value of experimental animal model testing will increase the likelihood that those findings can be more readily translated into therapies that benefit our patients," said Dr. K. Craig Kent, dean of the Ohio State College of Medicine.

Bias jeopardizes successful clinical translation of spinal cord injury research, Schwab said. Based on their findings, Schwab and his research team identifies tangible precautions to enhance the predictive value of experimental SCI research for a higher likelihood of translational success:

Reduce bias at the bench-side through by allocation concealment and blinded assessment of outcome; by reporting animals excluded from the analysis; providing a sample size calculation, inclusion and exclusion criteria, randomization, and transparent reporting of potential conflicts of interest and study funding.

Embrace and report negative or neutral results, which are persistently underrepresented in the public domain.

Re-use of published and experimental data augmented by large animal numbers (Big data repositories) through development of data-sharing models and international cooperation in preclinical SCI research to learn more about underlying differences.

"We conducted this study because we know that translation from animal models in clinical testing into human patients is difficult, but at the same time constitutes the main source of hope for many patients. As a physician scientist, I feel the need to improve this process as a basis for improving SCI care," Schwab said.