In 2008, one psychiatrist proved that the published antidepressant literature was missing a third of its trials — not through fraud, but through thousands of individually sensible career decisions. HEOR inherits that distorted record as model inputs, and quietly maintains a file drawer of its own. The fix Turner used generalises: find a record written before anyone knew the result.

Imagine you could see every trial that was ever run on a drug. Not just the published ones. All of them.

For most of medicine that's a fantasy, but in 2008 a psychiatrist named Erick Turner realised it was possible for one corner of it.¹ Before a company can sell a drug in the US, every pivotal trial has to be registered with the FDA — logged before anyone knows how it will turn out. The agency was accidentally holding something no journal ever has: a receipt for every trial, win or lose.

So, Turner pulled the receipts for twelve antidepressants. The FDA had 74 trials on file, and by their account the drugs were close to a coin flip: 38 positive, 36 negative or shaky. The journals told a different story. Of the 38 positive trials, 37 were published. Of the 36 failures, 22 vanished entirely, eleven were written up to read like wins, and only three were reported honestly. A diligent reader of the literature would see a 94% success rate, when the underlying record said closer to half.

But nobody saw those records. A scientist runs a trial, the result comes back negative, and writing it up is months of work that journals don't particularly want and tenure committees don't particularly reward. So the trial goes in a drawer, one perfectly sensible career decision at a time. Statisticians call it the file drawer problem, and I made a full video on the Turner story this week.

But the more interesting question for this audience is what it does to our own work. Because HEOR doesn't just read the publication-biased literature. We build on it, and we add a drawer of our own.

The inputs inherit the bias. Regular readers will remember this from the survivorship issue: your NMA is an analysis of trials that survived the editorial filter. The pooled effect size you carry into a cost-effectiveness model is inflated by construction — and rarely by the same amount in each arm, since older comparators have had more years for their failures to surface. Funnel plots, Egger's test, and trim-and-fill can tell you the survivors are biased. They can't tell you what the missing trials showed.

HEOR has its own file drawer. Think about how many cost-effectiveness analyses get built versus how many get published. The sponsor-funded model that came out unfavourable doesn't get written up. The conference abstract never becomes a paper. The country adaptation that flipped the conclusion stays internal. The published CEA literature skews favourable for exactly the same reason the antidepressant literature did — not fraud, just a filter at the point of publication. Anyone benchmarking an ICER against "published estimates" should hold that comparison loosely.

The Turner move generalises. What broke the problem open wasn't a better statistical correction. It was finding a record created before the filter could act. That's a habit worth stealing. ClinicalTrials.gov holds registered protocols and, increasingly, mandated results for trials that never reached a journal. FDA medical reviews and EMA EPARs contain trial-level data the publications left out. Clinical study reports, where accessible, are the fullest receipts of all. An SLR that stops at the published literature isn't a systematic review of the evidence. It's a systematic review of the survivors.

The diagnostic question is the same one Wald would have asked, pointed at a different filter: what would this evidence base look like if every study that was started had been reported? You can't usually answer it. But you can almost always check whether the registries, the regulatory documents, and the grey literature tell the same story as the journals — and when they don't, that gap belongs in your submission as a limitation or a sensitivity analysis, not in a drawer of its own.

Four hundred years ago, English philosopher Francis Bacon wrote that the human mind is more moved by affirmatives than by negatives. The journals are just Bacon's observation with an editorial board. The deciding skill, in evidence synthesis as everywhere else, is learning to ask what's not being shown to you.

— Mirko

Have you ever caught the file drawer in the act? A registry entry, FDA review, or CSR that told a different story from the published paper? Or sat in a meeting where a model was shoved into a drawer, maybe because the results weren’t quite supporting the commercial objectives? Hit reply & let me know!

Across 12 domains, the HEOR roadmap is a quick self-assessment against the 120 core skills of the field. Download it for free in less than 30 seconds, or spend 10 minutes customising it for a personalised breakdown of your gaps and what to work on next.

P.S. — Pharma Radar is written by Mirko von Hein. I help pharma and biotech teams navigate HTA submissions, cost-effectiveness modelling, and market access strategy across the UK, Ireland, and Germany. After a decade across IQVIA, Parexel, and Gilead, I'm now taking on selected engagements through Von Hein Consulting.