This incident points to more pervasive problems in the quality of the reporting of scientific findings in the media.
In my last article (Debunking 1 in 5 COVID-19 Patients Develop Mental Illness), I debunked a claim circulating around the world — that a Lancet Psychiatry study found 1 in 5 COVID-19 patients would become mentally ill in the first 3 months after diagnosis.
Researchers drew on the electronic health records (EHRs) from a US managed health care system to match 62,354 patients COVID-19 with patients with 6 other medical diagnoses.
Claims about mental illness were based on recording in the electronic health records of mental health diagnoses 14–90 days after diagnosis of COVID-19 or the comparison health conditions. These diagnoses of mental illness were originally made for administrative purposes — billing and documentation — and would be considered poor quality for scientific purposes.
I pointed out that the excess of mental health diagnoses associated with COVID-19 was almost entirely due to more adjustment disorders.
Patients diagnosed with COVID-19 were likely to be experiencing some anxiety (worried) or depressive symptoms (sad, pessimistic) that were not serious enough to justify a diagnosis of major depression or a serious anxiety disorder. So, a diagnosis of adjustment disorder was entered into their EHRs.
These do not seem like abnormal reactions to me. In fact, adjustment disorders were formerly called adjustment reactions and clinicians could not be reimbursed for treating them.
Dissenting from the portrayal of the study in the media
Why was my interpretation of the Lancet Psychiatry article at odds with almost all of the media coverage I could find, which proved to be remarkably similar?
I arrived at my conclusions by looking at the study design and closely examining the tables of numbers.
I applied knowledge that any journalist should have in writing about a topic that has great importance to readers facing a profound public health crisis like COVID-19.
America is facing a profound crisis of faith in democracy and in science. It desperately needs media filled with articles by journalists who think for themselves and consult experts when they need to.
In an ideal world, journalists should have called up their own independent experts and asked them about their impression of the study. Were these experts expecting a tsunami of mental health problems or did they think these authors might have overstated the importance of their findings a bit?
Journalists should have had the curiosity to ask how serious a mental illness an adjustment disorder and why there was not more depression associated with COVID-19.
In a bygone era, journalists were better able to play an important function in society. At their best, journalists buffered laypersons and policymakers from the self-promotion, conflicts of interest, and sheer overenthusiasm of researchers for their own work. Journalists provided independent assessments of science to satisfy readers’ curiosity, but also to guide their personal and collective decision making.
But journalists face different challenges than only a few years ago.
Most newspapers and magazines have laid off their full-time staff and depend on freelance journalists. Journalists can no longer assume researching articles will be reimbursed.
Often, journalists will not have the time to look at a scientific paper carefully or to consult their trusted sources. Few will have the courage or the recklessness to say that the authors of the Lancet Psychiatry study (from Oxford University, after all) are exaggerating the significance of their findings.
Editors will also not be pleased with a story on which they cannot slap a clickbait, screaming headline like “1 in 9 mentally ill!!”
Media have gone online, some abandoning print editions altogether. They need clickbait to attract and retain readers’ interest. Traffic to a media website and readers’ time spent there are tracked as altmetrics, which determine the advertisements the media get and what they can charge for ads.
This consideration matters for which articles from which journalists are published.
Science Media Centre of London (SMC_L)
Media coverage of the Lancet Psychiatry article was almost identical worldwide. If articles quoted an expert, it would likely be identical quotes from the same experts. A lot of the articles credited Reuters News Service. The articles that did not credit Reuters but named a journalist in the byline were generally quite similar, even having the same quotes.
Noting that some authors of the Lancet Psychiatry article were at Oxford University, I immediately suspected the Science Media Centre of London (SMC_L) was involved.
When science hits the news agenda, it’s our job to pass on to journalists as much accurate information as we can, as quickly as possible. In order to do this, we send out quotes from experts and statistical analyses of scientific studies, in addition to running regular press briefings on the latest hot topics. Find our most recent Roundups and Rapid Reactions, briefings and ‘Before the Headlines’ analyses below…
Who are the experts that SMC_L consults? Can anyone submit their credentials and expect to be called?
We are always on the lookout for top experts, but it’s important that each scientist knows us before we call them. So, we generally don’t add scientists before we’ve worked with them at least once.
It sounds like British English for “Don’t call us, we will call you.”
SMC_L warns of storms around science stories in the media.
When science hits the headlines, it’s not only journalists and scientists who find themselves in the eye of the storm, but also press, media and communications officers, working to support their scientists and engineers whilst also providing journalists with what they need.
I expect that bad science will get into the media and even good science will get misrepresented. Such occurrences are inevitable. When it happens, controversy and criticism are healthy and necessary parts of fixing things.
You might get a sense that SMC_L disagrees. They seem to be offering shelter from such “storms.”
Expert reaction to study looking at psychiatric diagnoses after COVID-19
I don’t like authorities telling me how to think or what to write, even if, like Ringo Starr, they can call themselves Sir or Dame or wear medals at breakfast.
Searching for the Lancet Psychiatry study, I found a past Briefing and an Expert Reaction with seven of SMC_L’s usual suspects as experts.
Six of the seven mostly sounded like experts, but really did not get into the nitty-gritty of the study, content enough to demonstrate expertise.
One expert had “Sir” between “Prof” and his name. Another had “Dame” between “Prof” and her name.
I cannot understand why Brits bring up knighthood and damehood in these contexts. When Ringo Starr got to call himself Sir, it did not improve my appreciation of the Beatles.
There was not very much reference in this Expert Reaction to crucial methodological details of the study and the limitations of EHRs as scientific data.
It would seem that a grasp of these details should temper any recommendations for clinic or public health policy based on the study.
No expert conveyed any sense of the messiness of big data from an American health care system, especially if one is intent on making policy recommendations for organizing a response to mental illness associated with COVID.
Six of the Expert Reactions sounded like a string sextet accustomed to playing together, mostly without creating a fuss. The seventh, however, arrived like a tuba player, oblivious to not contributing to the harmony.
Prof David Curtis politely alluded to a lot of problems with the Lancet Psychiatry study. He seemed to wonder aloud why such a fuss is being made about it. He injects some uncommon common sense into the discussion
“It may be unsurprising that this happens a bit more often in people with COVID-19, who may understandably have been worried that they might become seriously unwell and who will also have had to endure a period of isolation.”
If there is any doubt about where he stands, he clarifies at the end:
“Overall, the results reported seem broadly plausible but I’m not sure that they have specific implications for patients or health services.”
My kind of expert. I hope Prof Curtis gets invited back by SMC_L.
I think my humility is still intact after decades of writing hundreds of scientific papers, some that deserve more criticism than they received. I welcome criticism from experts. I welcome peer-review by academics, but also feedback from patients who are experts on their experience and who have had to live with the modest clinical and public health implications of what I have written.
Yet, I think there is a vital distinction to be drawn between expertise and authorities. The good experts have the credibility to gain our attention by showing things we might not know, but that we would agree we should. The not-always-good authorities believe they have the power to enforce rules and tell us how to think and what to write.
I think that is what SMC-London is doing here. I am biased by my experience with them, but that bias is valuable because it is grounded in experience. I have seen their ways too often for my liking.
- One of the experts being cited in this Expert Reaction — the Dame — also endorsed an awful study that she claimed showed cognitive therapy for psychosis was effective for psychosis in the absence of medication. The study did not show that, but such claims encouraged some patients to refuse medication. The Cochrane Collaboration showed twice in systematic reviews that cognitive behavior therapy was ineffective.
- One of the experts who was cited — the Sir — advocated cognitive behavior therapy and graded exercise for chronic fatigue syndrome, which patients insisted was ineffective and harmful. Fiona Fox, Director of SMC_L organized a campaign that media that condemned the patients as militant terrorists. The press releases were picked up in British media. The patients were subject to considerable abuse and harassment as a result.
Whiskey, tango, foxtrot! The patients were often bedbound or confined to wheelchairs. I do not think anyone would recruit them as suicide bombers or even assassins.
- I decided to look into the matter. I requested data from the PACE clinical trial for reanalysis. I believed the PACE investigators were legally bound to make these data available. Fiona Fox, Director of SMC_L organized a letter-writing campaign to Parliament calling for researchers to be exempt from such requests.
As promised, SMC_L tried to provide shelter from the storm.
- More recently, patients with chronic fatigue syndrome and professionals from around the world convinced the UK National Institute for Clinical Excellence to issue draft recommendations that no longer endorsed GET as a safe and effective treatment. SMC_L has organized resistance from academics strongly invested in this treatment, including the Sir.
I am exceptionally lucky. I came out of public housing on a shitty mudflat in Chelsea, MA, and spent my high school years on welfare. By many accidents of circumstances, I somehow ended up an emeritus Ivy League med school professor.
I am painfully aware of my deficiencies in preparation and seek my corrective expertise where I can find it. But I don’t like authorities telling me how to think, even if, like Ringo Starr, they can call themselves Sir or Dame or wear medals at breakfast.
America is facing a profound crisis in its faith in democracy and in science. It desperately needs media filled with articles by journalists who think for themselves and consult whatever independent expertise they feel they need.
I don’t care that SMC_L gives lots of stuff away free and does not mind if journalists plagiarize them.
Journos, please find your own experts and write your own articles on which you can proudly stick your byline.