Most medical studies don’t report their results – a new UK law could change that.

This is a science fiction starring Stuart Ritchie, a newsletter for i. If you’d like to receive this straight to your inbox every week, sign up here.

Historical records are full of holes. Anyone who is interested in history knows this: people die without having recorded their memories; Books and records are deliberately destroyed by conquerors or those who wish to forget the information they contain; Helmets, shields, spears and other items will rot if we’re not lucky enough to survive. Even the records that we Do written from various preconceived points of view, often deliberately omitting facts that are inconvenient for the author.

Here’s the thing: historians have to work their way through this half-Swiss cheese board and figure out what really happened. With the disorder of human history, it cannot be otherwise: history is full of holes a-priory.

Scientific evidence, however, is not a-priory A scientific report could theoretically be a perfect record of all the research and experiments ever done. We can look back and find all the studies that have ever been done on this phenomenon and get a complete picture of the evidence.

And yet the scientific record is also full of holes. At this point, it’s perfectly normal for scientists to run an experiment, check the results, and just not publish the study for whatever reason. Perhaps they didn’t like the results—they didn’t confirm their favorite theory, or they didn’t show that their favorite treatment worked. Maybe they were too complex and they didn’t know how to write them coherently. Maybe they were just busy with other things and never bothered to write a paper.

For all these and many other reasons, the scientific literature turns out to be a very incomplete description of the research that has actually been done. And just like the story, what remains can be a distorted picture, biased towards positive, exciting, and headline-worthy results. This puts future researchers—and anyone who reads the scientific literature—at a huge disadvantage, as they get a skewed view of research and may end up wasting resources chasing phantom results or even using medical treatments that don’t work or dangerous. .

Even in one of the most regulated areas of research—human medical research—scientists publish all their research poorly.

All clinical trials must be registered before they start: governments, including in the UK, have websites where you must upload all documents and plans after you have agreed to start the trial, but before data is collected. Most medical journals won’t let you publish your research if you don’t.

But these research notes give us an idea of ​​how much research is being done… and then disappearing into the air. In 2015, the US Investigation Registry analyzed thousands of medical studies conducted between 2008 and 2012. Only 13.4 percent of them were published within a year of completion, and only 38.3 percent were published even after several years. . The study continued until 2013, after which many studies were not published. 5 years; It is believed that many of them have simply never been published.

Therefore, it is extremely important that scientists – whether at universities or pharmaceutical companies – publish their results. But that would be Real well if we could post them fast – i.e. within one year after the end of the procedure. Worst of all, the pathetically low 13.4% is pathetically low. Despite legal requirement in the US (as of 2007) to publish everything within 12 months.

It turns out that the figure “13.4 percent” is probably an underestimate. A 2020 study found that the US clinical trials website did not have enough information when the first study was completed and that the researchers ended up including many studies that were not scheduled for publication for whatever reason (perhaps this is there have been studies in which, among other things, longer follow-ups were planned). But this recent study itself showed that the corresponding number reported 40.9 percent during the year. This is much better than 13.4 percent, but still well below what we would like to see.

The UK may provide an answer this month. In response to an expert consultation on clinical trial registration, the government said it would legally require the Medicines and Health Products Regulatory Agency (MHRA) to report all trials within 12 months.

You might say, “Well, there are legal requirements in the US too, and they’ve been in place since 2007, which is what you just told me. majority the scientist merrily breaks it!” But the difference is that the MHRA is – at least in theory – allowed to do so. punish rule breakers.

They state: “Failure to comply with the new requirements for the transparency of training … is expressly stated in the legislation as the reason for the rejection of the application for approval.”

This means that if you do not report your legal proceedings within 12 months, the government may prevent you from conducting further investigations in the future. For many doctors, this would mean the end of their careers. For pharmaceutical companies, this would make it difficult to market their drugs.

Transparimed, a website advocating for better clinical trial registration and reporting, says the new law is “light-years ahead of the UK compared to the United States, where the US Food and Drug Administration [Food and Drug Administration, the US’s regulator] has not yet sanctioned medical companies or universities for violating the Results Reporting Act of 2007. It’s hard to disagree: it’s amazing that none of the authors of the roughly 60 percent of studies who didn’t report their results within the 12-month period required by law were penalized (reminds me of my previous newsletter about scientific fraud and how we don’t punish them properly).

There are also many other reforms in the government’s announcement to improve data sharing, allow for more flexibility in terms of the specific level of consent needed for a study when it is very low risk, studies with similar treatments are linked and companies cannot get permission. to conduct a test and then simply do not bother to do it. All in all, a big step forward.

It may be depressing that we did not have such an agreement before. After all, we spend huge amounts of taxpayer money on medical research, and it often turns out to be much less useful than it could be (if the results are published many years later) or completely wasted (if the results are never published at all). It’s a slow process, but building a solid system that actually encourages scientists to do the right thing will vastly improve scientific data and prevent it from becoming too similar to historical data.

Other things I’ve written lately

As you know from a newsletter published a few weeks ago, I like to think about the weirder aspects of psychology, like the study of psychic powers, spirits, and so on. The new book isn’t really about parapsychology, it’s about why people sometimes feel “presence” in a room with them. I wrote a review/statement about it.

This week in the news was a Scot who raped a 13-year-old girl in a park when he was 17 but was sentenced to community service rather than jail. This is partly because the new Scottish Prison Rules stipulate that anyone under the age of 25 must be treated as if their brains were not fully developed, preventing them from being fully guilty of a criminal offence. I think this idea, obviously based in psychology and neuroscience, doesn’t really work, and I’ve explained why here.

Science Link of the Week

If you’re interested in cutting-edge technology, this Substack post on the history of DARPA and its amazing advances in neuroscience is a great read. This is an incredible list of technologies that have pushed the boundaries and helped patients with all kinds of brain diseases. I hope that the British version of DARPA, the new organization of ARIA, will be able to make similar breakthroughs in the coming years.

This is a science fiction starring Stuart Ritchie, a newsletter for i. If you’d like to receive this straight to your inbox every week, sign up here.