NEW YORK (Reuters Health) - Skimpy reporting by medical journals has left German doctors trying to write psychiatric guidelines in a pickle: they don’t know what studies to trust.
After reviewing 105 supposedly ‘gold standard’ drug trials for bipolar disorder, they had to exclude several of them and downgrade even more because critical information was lacking from the reports.
In the end, only six of the studies passed muster with the review panel.
“When we looked at the data, we found it wasn’t so easy to decide what studies we could rely on,” said psychiatrist Dr. Andrea Pfennig of Carl Gustav Carus University Hospital in Dresden, who sat on the panel. “We were really surprised.”
Bipolar disorder, sometimes called manic-depressive disorder, is a mental illness that causes severe mood swings — from depression to manic excitement.
The side effects of the drugs typically used to treat it — such as mood stabilizers and antipsychotics — can be serious, so guidelines are important.
Dr. Daniel Strech, another panel member who studies research ethics at the Hannover Medical School, said the findings are concerning.
“One could question whether doing trials with thousands of patients was really worth it if you cannot use the data,” he told Reuters Health.
Their quality assessment, which appears in the Journal of Clinical Psychiatry, spans all so-called randomized controlled trials published between 2000 and 2008.
Such trials are considered the strongest tool researchers have at their disposal to test the benefits and harms of new treatments. Patients are randomly assigned to the new treatment or a comparison group — for instance a dummy pill or a standard treatment — to make sure any effect observed can be chalked up to the new drug alone. And it’s best if the investigators don’t know which patients are in which group, so they won’t subconsciously treat some patients differently than the others.
But the devil is in the details.
If the randomization process isn’t properly done, for instance, or investigators in a trial become aware of the type of treatment patients are receiving, that could sway their interpretation of the results.
To judge how well such information was reported in medical journals, the German researchers used a checklist with 72 items that should be included in reports.
They found a quarter of those items weren’t adequately addressed in the 105 trials of bipolar disorder drugs.
As a case in point, only 15 percent mentioned how they had masked the type of treatment patients got. Earlier research has shown that failure to conceal that process could lead to exaggerated drug claims, Strech said.
“The reporting is often more positive,” he said. “As a doctor, you have to consider that maybe the treatment is not as effective as reported.”
In most cases, information was also lacking on important measures such as how many patients would have to be treated in order to see improvement in a single patient or, conversely, to see a side effect.
While the poor reporting is a stick in the wheel for experts trying to settle on the best treatment for mental illness, exactly what it means for patients isn’t clear.
“We can only say that, from the publications, we can’t tell if the data are good or not,” said Pfennig.
In the meantime, without enough good data, the German guidelines panel has still not been able to issue clear recommendations on how doctors should best treat their patients with bipolar disorder.
SOURCE: bit.ly/hP0SCt Journal of Clinical Psychiatry, online January 25, 2011.