Gaming Measurement of Vaccine Efficacy
Gaming Measurement of Vaccine Efficacy
Biostatistics Professor and data science blogger Jeffrey Morris writes,
We (meaning those of us who are paying attention to the vaccine studies) are all aware of the Kaplan-Meier curves that show that antibody buildup seems to ramp up to effective levels at around 10 days after vaccination. And yet, clever as the hivemind may be, none of us has yet devised a way to vaccinate people in such a way that the first 14 days don't actually happen.
The problem is that when discussing the most important aspects of medical intervention (such as risk-benefit analysis), starting measurement of efficacy at that point makes no sense. But it may not be clear why it doesn't make sense to some people, and bias may get in the way. In order to sort out such temptations to reset basic modeling principles (like to pretend away the first 14 days because maximal efficacy doesn't yet take place), the experienced mathematician or statistician often starts with the extremal principle in order to assess the difference. That is to say that it should be a professional habit to examine a case in the neighborhood of the worst possible scenarios, and then to consider the spectrum of results in between the relevant boundaries.
The Morris Therapy
Since Jeffrey seems to feel okay with simply declaring (as he has in numerous emails we've shared) something like, "I'll be fine with including the risks when they're published in a peer reviewed journal" (you know...like the Surgisphere paper), while feeling just fine about performing computationswithout asterisks pointing out that his calculations would look wildly different depending on that suppressed (autopsies) data, I'll honor him with the naming of this edge-case.
Suppose that tomorrow it is announced that in a Wuhan laboratory---located somewhere between the French-designed Wuhan Institute of Virology and the fabled wet market---that an NIH funded project results in highly valuable intellectual property in the form of the Morris Therapy. After rigorous mandatory testing of the Morris Therapy on Uyghurs, including the high risk elderly, pregnant women, and also children, the Morris Therapy demonstrates 100% efficacy in preventing COVID-19...after day 13.
NIAID Princeling, Dr. Anthony Fauci quackly announces an EUA both for the Morris Therapy, and also swift approval of the as of yet unavailable COMorrisY Therapy. Nobody was reached at any governmental organization who could explain which of these has indemnity, and whether citizens pay for those liabilities by giving up their children as they drop them off at school where they are to be treated by swiftly trained gym teachers.
During the first two weeks, 80 million Americans jump at the opportunity to receive Morris Therapy. However, 79.2 million of those Americans seem to be...well...missing. With doctors and morgue owners tight lipped, the CDC reports that indeed, the Morris Therapy has resulted in 100% efficacy in preventing COVID-19 after the first 13 days. Scientists and other people who pay attention to things rush to post videos on YouTube, Twitter, and Facebook pointing out that there are nearly a million reports of death in the Highly Profitable Therapy Adverse Events Reporting System (HPTAERS). Fact checkers [just doing their job] point out that causality of those deaths HPTAERS hasn't been proved, and censor all those reports. University faculty petitioned to silence, reprimand, or simply harass their colleagues who participated in those reports, labeling them with the pejorative "anti-Morrisers". No autopsies are performed on the 79.2 million Americans who died during the first 13 days after a dose of Morris Therapy.
What Just Happened?
What happened is that 99% of the people who received the Morris Therapy dropped dead during the first few days. Even worse, many were taken to the hospital, entered into a database as non-Morrised, and as hospital beds filled up, media outlets declared a "pandemic of the un-Morrised". All the social pressure makes it hard to talk about the problem. Meanwhile, nobody does autopsies on the bodies that might reveal clues to the deaths, such as the presence of spike-Morris protein in organs all over the body. All the 79.2 million deaths are presumed to be COVID-19 deaths, and the media dedicates itself to a 48 hour marathon of fear porn, lamenting that poorer nations are not receiving their equitable share of limited supplies of Morris Therapy.
Meanwhile, health officials and all those devoted to the success of the Morris Therapy as the final solution to the COVID-19 pandemic…"correctly" point out none of the 800,000 survivors of the Morris Therapy have COVID-19 (though nobody really wants to talk about disease etiology as it might link Morris Therapy as a Type II COVID-19). In the end, the entire population of the SARS-CoV-2 virus decided as a "species" (if we can call a virus a species), that invading the U.S. just wasn't worth it at all, deciding instead to go live amongst the flies, minks, bats, and pangolins.
Yes, The Morris Therapy is an Extreme Case
But it also displays the point: if you're computing without respect to what happens during the early period---regardless of expectations of efficacy during that time (that passing of time that cannot be avoided)---then you are not examining the sum total of the effects of the therapy.
And that's just plain dumb.
It's so dumb that I've found it extremely hard to tell the difference between those in or around public health who are dumb and those who are simply of bad faith. I'll leave that up to readers to judge.
The Statistician's Solution
A statistician should never be satisfied with an important, policy-driving computation that puts such a blunt thumb on one side of the scale. The ethical goal should always be to match numbers most meaningfully to reality. It is fine to define something like an interval efficacy number, but to the extent that we do, we should communicate clearly to the public that (1) this does not express the efficacy on the superset of time from the outset of treatment, and (2) that no efficacy numbers (even 100%) can possibly tell us whether a therapy has a positive risk-benefit. The responsible thing to do is to downplay such an efficacy number, and demand that the work be done to achieve meaningful data that would allow for the computations necessary for an even modestly meaningful risk-benefit analysis.
Comments
Post a Comment