You might think that questions about what was found in the clinical trial published in Nature Medicine could be resolved by simply checking the Results section of the paper.
Many people do not inspect what is reported before forming an opinion or accepting someone else’s opinion. The actual results are certainly worth a look.
But I think we need to consider whether the authors even were asking the right clinical question.
By narrowly construing the trial as a test of the efficacy of MDMA for PTSD, the authors may have missed some limitations. The trial is NOT primarily being a drug trial, but more properly, a trial of a drug always being delivered in conjunction with a poorly specified and unproven psychotherapy.
I have some relevant background and expertise for forming an opinion about this.
I was involved in various stages of writing and reviewing grant proposals, applications for use of human subjects, and justifications for the burden that studies place on medical patients. I was a co-principal investigator on a center grant with the responsibility of developing and implementing psychosocial interventions combined with drugs. I served on the committee monitoring the progress of a major clinical multisite trial of treatment of depression to prevent heart attacks for safety and data quality. I was an external scientific advisor conducting process evaluations that might reveal what happened in trials for anomalous results.
I have never encountered anything at all like the psychotherapy manual for the Nature Medicine trial of MDMA for PTSD.
If you like, you can wade through the actual manual here, but it is very vague and demands of your time to get to crucial details. Or you can see how promoters describe the treatment in promotional material on their websites, including graphic videos.
I am a clinical health psychologist concerned with delivering the best treatment within the limits of scarce funding. That makes me a mental health services researcher. I am deeply concerned with the lack of assurance that the same psychotherapy was delivered across settings in the trial reported in Nature Medicine.
I could imagine making a bit of a fuss if I were on a committee to evaluate ahead of time whether a major clinical trial of MDMA, a drug that was illegal and otherwise unpatentable could depend on the validity of this treatment and whether it was faithfully delivered.
Basically, investors are asking for a monopoly because they will ensure the safety and efficacy of MDMA by embedding it in a package with their psychotherapy. They claim to be so convinced by a lack of evidence that MDMA is safe or effective outside of such a package. So much so that they will not bother to gather new evidence against the null hypothesis of MDMA being unsafe and ineffective for PTSD.
In the loose, informal atmosphere that occasionally develops in closed meetings, I could imagine my younger self reading juicy sections from the manual, and insisting on playing promotional videos to the committee from the 15 sites that were available to prospective participants during recruitment.
With some degree of mock seriousness, I might have asked whether different music might have elicited a different response from patients, whether the flowers and fake artifacts were necessary.
What if a patient did not welcome what was intended to be the soothing touch of a same or different gender person while under the influence of a disorienting drug and while possibly being stimulated to sexual arousal?
I could expect blowback and protests if I raised the last point. I would come prepared with documentation that sex with a patient had occurred in at least one MAPS trial and that the unlicensed counselor had pleaded, of course, that the sex was consensual.
This claim is outrageous without substantiation. I invite you to find it here. Psychedelic therapy has a sexual abuse problem
I doubt anyone involved in reviewing the data from the trial for the US Food and Drug Administration has ever seen such psychotherapy being proposed to accompany all marketing of a drug. I would love to hear the discussion the review will generate.
Claims that findings are groundbreaking, breakthrough, or otherwise extraordinary deserve more scrutiny and maybe even skeptics thinking longer and harder about what was going on in a study.
Many on social media assume that claims that have made it into a world-class medical journal no longer need to be vetted. They believe that they can take what is said about the psychotherapy in an article in an impressive journal with their critical tools to appraise the study being left in their toolbox.
I am not one of those people. I am hoping to convince some readers that they do not want to be that kind of person either. That is generally my goal in teaching and writing about clinical trials, but I think that the Nature Medicine article is a particularly good teachable moment. The authors and their backers are screaming so loudly and incessantly that there has never before been such a study.
I am getting some pushback to my skepticism about the trial, only some of which will be disclosed here. There has been an effort to stop the publication of my critiques and remove what I have published so far. If what I have written were patently stupid, I think that would be discovered soon or later and what I have published could be left published to embarrass me.
The threshold should be high for making an article disappear, rather than simply flagging with a statement of concern or retracting it.
I first probed a fawning New York Times article about the trial with improbable claims, obviously written with the collaboration of the Nature Medicine authors. I complained that the journalist should have protected readers by getting her own experts and by providing an independent critical appraisal.
Next, I limited myself to a single sentence in the abstract of the Nature Medicine article. I wondered aloud why so few “experts” were speaking out about a reporting of this study that was so wrong in so many ways.
In the last installment, I focused on only the list of authors, the contributions that justified their getting authorship, and the authors’ affiliations. I concluded that there was good reason to treat the study not as strictly as a pharmacological trial but as a trial of psychotherapy in combination with either a drug or an inert pill-placebo. I proposed:
This reframing involves a very different set of criteria that could expose a different set of inadequacies in the trial of MDMA as a treatment for PTSD — such as the lack of standardization of the therapy across settings and therapists.
I am pleased that my intuitions have generally been confirmed that there is something wrong going on with the way the study is being sold. Yet, I wince at some of the things I have written and mistakes that may have been made. Even if I have been wrong in some respects, I could be on to something. We could reach a consensus that the study is flawed, at least enough to temper the authors’ claim made in the abstract and echoed all over the world:
“We conclude that MDMA-assisted therapy represents a potential breakthrough treatment that merits expedited clinical evaluation.”
The authors and their sponsors are pushing for expedited review and approval by the US Food and Drug Administration. This was hastily done with intravenous infusion of ketamine, which — as I covered here — did not go so well.
One reader,Julian Willet, MD is unconvinced enough by what I have been writing to post a comment:
Nature Medicine is a well-respected journal and generally decently trusted (though naturally, some papers can get through the cracks). Perhaps that occurred, but I cannot say. I skimmed the paper and the results are significant/impactful, which would make it an attractive paper to publish even if there could be potential controversy.
I asked the opinion of an expert statistician who has consulted on trials of psychedelics and he replied:
*I have read the results section a few times since published (and compared) to protocol. I would say that generally it looks sound and I would guess they have a professional statistician in the team or use a consultant. That’s from what I can deduce from the paper.
His comment left me self-doubting and defensive. I went back to the Acknowledgment section of the article. I counted the number of therapists listed. I replied to him, “What if there are 40 therapists?”
Suffice to say, that reply changed his mind and got him doubting too.
Only as I sat down to write this article, did I realize the implications of what I had written to him.
The Nature Medicine article portrayed the clinical trial as evaluating whether MDMA was superior to an inert pill-placebo under tightly controlled conditions. It would be reassuring to know if the results did not vary across the clinics and providers, but we probably should not expect any surprises.
It is simple enough to standardize the providing of a pill and not expect much variation across sites and providers. Not so with psychotherapy that is so incompletely specified. I expect lots of variation in how the psychotherapy is implemented with which patient, with which therapist, in which setting.
The trial is initially blinded so that neither the clinician nor the patient knows whether the patient is assigned to get the MDMA or an inert substance in an identical capsule. The trial quickly becomes unblinded in the first half-hour of the first eight-hour session. There are three such sessions. I would be especially concerned with the improvisation that would occur once the patient and therapist knew whether the patient was getting the MDMA.
I do not have to prove my skepticism is correct. The burden is on the authors of the Nature Medicine article to prove me wrong. They are facing an evaluation by the FDA with a lot at stake.
Stay tuned and we will probe the Results section, aided by the lens I have developed in this article.
