Study Concludes: Most Studies Are Wrong.

n/a

As the number of scientific studies exponentially mount, surely we are advancing scientific inquiry at an ever increasing rate.  But perhaps we are simply increasing the scientific “noise.”

Enter the Reproducibility Initiative, which will try to reproduce your findings for you by an independent lab.  Yes, we’ve come to a point where you will need to pay to have the results you think you have confirmed by someone else.

Consider the NewsDaily’s article that “Bayer Healthcare reported that its scientists could not reproduce some 75 percent of published findings in cardiovascular disease, cancer and women’s health.” Or that “Amgen reported that when the company’s scientists tried to replicate 53 prominent studies in basic cancer biology, hoping to build on them for drug discovery, they were able to confirm the results of only six.”

How is that possible?  Don’t we have scientists dedicated to publishing whatever results occur?

Anyone who remembers science class knows the answer.  When you got the results you expected, you didn’t go over the equipment and the method with a fine toothed comb.  You assumed you did the experiment right and turned it in.  Only when you got wildly odd results that didn’t agree with what you were looking for in the slightest did you go back over your method and equipment to find the error.  Even if you had the highest ethics, it would be perfectly possible to miss some error as long as the results fell into a “reasonable” outcome.

Having more people do the same testing can lead to better results, but if you are all testing in the same area and watching one another’s results, chances are good that you started looking for the same results in your experiments.  A friendly classmate might even help you by telling you how to change your equipment to get a desired result.  So more tests do not necessarily lead to more accurate results.

But don’t believe me.  Have a gander at the most read PLoS article ever, entitled: Why Most Published Research Findings Are False.

In this appetizing little mathematical jaunt, the author takes us down the reality that false positives are far more likely than finding the truth.   Even before we add in publication bias, tenure track pressures, and financial incentives, it is just too easy to find the results you’re looking for.  The author Ioannidis states:  “manipulation could be done, for example, with serendipitous inclusion or exclusion of certain patients or controls, post hoc subgroup analyses, investigation of genetic contrasts that were not originally specified, changes in the disease or control definitions, and various combinations of selective or distorted reporting of the results. Commercially available “data mining” packages actually are proud of their ability to yield statistically significant results through data dredging.”

So just how many of us are taking drugs created for an illness, supported by studies created to support that drug’s ability to treat that illness, and prescribed by doctors who believe that the drug will effectively treat our illness despite all of our claims that the drug really isn’t working?  Meanwhile we as patients want to have something that works for our illness, so we spend a lot of time giving the drug “time to work” when it really never does anything to help us.

As someone who works in the alternative healthcare field, I don’t know whether to laugh or cry.  It feels like the rug just got pulled out from under all the work we’ve done to start bringing the field up to the standard of scientific inquiry.  Suddenly what was clinically relevant information is in question, and the standard drugs that we’re trying to compare to the alternatives are also in question.  How do we know what works?

Fortunately, I’ve got an ace up my sleeve.  I’ve been working with ornery, independent minded patients for years who don’t mince words when things don’t work.  So I’ve got an ongoing practice based on what is working in the field, using my patients as my resources.  Maybe it is time for all doctors to use their patients, rather than the drug reps, as their resource for what really works.

Related articles

• “No.. Not Yet”:Answering the Patient’s Request for “Off Label Use” of a Drug (bioethicsdiscussion.blogspot.com)
• Shoring Up the Mantra of Science: Take Nobody’s Word for It (reason.com)
• Why scientists seem to change their minds (kirstyevidence.wordpress.com)
• Good Scientist! You Get a Badge. (slate.com)
• R&D experts applaud new initiative to validate biomedical studies (fiercebiotech.com)
• Glaxo’s Experimental Drug Halves Asthma Attacks in Study – Bloomberg (bloomberg.com)
• Experimental cancer drug makes mice infertile without side effects, scientists claim male birth control discovery (cbsnews.com)
• Amgen Halts Pancreatic Cancer Trial After Drug Failure – Bloomberg (bloomberg.com)
• Study off Mass. coast finds noise harming whales (bostonherald.com)
• Trusting the science (scienceornot.net)

http://humanbodyengineer.wordpress.com/

Source: https://humanbodyengineer.wordpress.com/2012/08/17/study-concludes-most-studies-are-wrong/