Our previous post, Antidepressants: The Evidence, offered a quick rundown of what three major depression treatment guidelines tell us and what they don’t tell us. The guidelines were issued by the American Psychiatric Association (APA), the Canadian Network for Mood and Anxiety Treatment (CANMAT), and (in the UK) the National Institute for Clinical Excellence (NICE). All three guidelines cite strong evidence in support of the efficacy of using antidepressants to treat moderate-to-severe depression in both the short and medium term. But is that evidence trustworthy?
In this post, we will examine the evidence for short-term treatment:
Randomized double-blind controlled trials (RCTs) are the gold standard of medical treatment research. As a general rule, a pharmaceutical company seeking FDA approval to market a drug for a specific condition needs to submit two successful Phase III RCTs demonstrating the safety and efficacy of its product.
By Phase III, a drug company is reasonably confident of success. Previous rounds of studies - in the lab, on animals, on small groups of humans - have picked the winners and weeded out the losers. The costs for developing a new drug are staggering, in the billions. (Check out this article in Forbes.) The pay-off is equally staggering - billions and billions more. The drug company (more specifically its intermediaries) recruits a large patient population. A million things can go wrong. Everything has to go right.
An RCT “randomizes” subjects into two similar groups, the test group who receive an active agent and the control group who are given a placebo. The front-line clinicians and researchers are blind to both the status of their subjects and the pills they are administering. The testing begins.
An antidepressant trial generally lasts six weeks. The status of each subject is monitored weekly. A common measure is the HAM-D rating scale. A successful result is at least fifty percent of the test subjects getting at least fifty percent better on the HAM-D. The test drug also needs to outperform the placebo by a substantial margin. The difference between the test drug result and the placebo result is referred to as “effect size.”
Sounds straightforward enough, right? The NICE Guideline points out some of the drawbacks in taking these studies at face value. For one, studies sponsored by drug companies (the vast majority are) are four times more likely to demonstrate positive effects than independent studies. Also, medical journals usually only publish successful studies. Thus, any meta-analysis based on published studies is flawed from the get-go.
In addition, high drop-out rates (between 20 and 35 percent for antidepressant trials) can skew a study in either direction. Excluding the drop-outs tends to leave only the responders in the study, which virtually insures a favorable result. Including the drop-outs guarantees failure. A statistical device known as “last observation carried forward” attempts to compensate, but instead of a clear result we find ourselves placing our trust in statisticians.
Antidepressant studies are also plagued by high placebo results (generally in the 40 percent range). The NICE Guidelines cite three meta-analyses of Phase III RCTs (including unpublished studies) by Irving Kirsch that found little real difference between the antidepressant result and the placebo result. (The APA and CANMAT both ignored the Kirsch studies.)
In defense of high placebo results, NICE noted (with an apparent straight face) that the populations in RCTs do not comprise a true sample in the first place. This is an old argument based on the startling admission that researchers essentially cheat to get patients into studies. Thus, patients with mild depressions are represented as having severe depression. Naturally, the results are off.
This begs the obvious question: Would the results be better if we actually had some properly conducted studies? If so, where are these studies?
Even supporters of antidepressants acknowledge a virtual dead heat between the antidepressant and the placebo. My own search came across a meta-analysis that appeared in the May, 2000 British Journal of Psychiatry. The study tabulated the results of 69 RCTs (including unpublished findings) that tested Prozac for depression. Two of the study authors were employed by Eli Lilly. The result? “Fluoxetine was superior to placebo but effect size was low.”
Perhaps if the bar for a successful antidepressant trial were raised (say to 60 percent of patients getting 60 percent better) we would not have to worry about the test drug beating the placebo. Then again, there would be no approved antidepressants on the market.
To sum up, antidepressant RCTs consistently produce results showing fairly modest improvements in depression scores, where the test drug beats the placebo by a whisker. These findings are confounded by high drop-out rates, non-representative patient samples, and an over-reliance on statistical devices.
Does this sound like the “strong” evidence cited by three treatment guidelines in support of the efficacy of antidepressants? I would respond with a resounding “no.”
Having said this, RCTs measure groups, not individuals. They record a mean or an average, not a spread of results. A true result spread would show a substantial minority doing spectacularly well on an antidepressant. Unfortunately, we have no way of knowing in advance who that minority is. You might want to ask yourself:
- Is a one-in-three or one-in-four chance of a substantial improvement worth trying an antidepressant?
- Is a one-in-two chance of a modest yet significant improvement worth it?
- Is a fairly good chance of a slight improvement worth it?
You are the true experts. You tell me. Comments below ...
Coming soon: Antidepressants for the Medium and Long term - How Strong is the Evidence?
Published On: August 31, 2013