Vox asks What Went Wrong With The Media’s Coronavirus Coverage? They conclude that the media needs to be better at “not just saying what we do know, but what we don’t know”. This raises some important questions. Like: how much ink and paper is there in the world? Are we sure it’s enough? But also: how do you become better at saying what you don’t know?
In case you’ve been hiding under a rock recently (honestly, valid) the media not only failed to adequately warn its readers about the epidemic, but actively mocked and condescended to anyone who did sound a warning. Real Clear Politics has a list of highlights. The Vox tweet saying “Is this going to be a deadly pandemic? No.” Washington Post telling us in February “Why we should be wary of an aggressive government reponse to coronavirus (it might “scapegoat marginalized populations”). The Daily Beast complaining that “coronavirus, with zero American fatalities, is dominating headlines, while the flu is the real threat”. The New York Times, weighing in with articles like “The pandemic panic” and “Who says it’s not safe to travel to China”. The constant attempts to attribute “alarmism” over the virus to anti-Chinese racism. Etc, etc, etc.
One way people have summed this up is that the media (and the experts they relied on) did a terrible job predicting what would happen. I think this lets them off too easy. (...)
A few weeks ago, I wrote a blog post on face masks. It reviewed the evidence and found that they probably helped prevent the spread of disease. Then it asked: how did the WHO, CDC, etc get this so wrong?
I went into it thinking they’d lied to us, hoping to prevent hoarders from buying up so many masks that there weren’t enough for health workers. Turns out that’s not true. The CDC has been singing the same tune for the past ten years. Swine flu, don’t wear masks. SARS, don’t wear masks. They’ve been really consistent on this point. But why?
If you really want to understand what happened, don’t read any studies about face masks or pandemics. Read Smith & Pell (2003), Parachute Use To Prevent Death And Major Trauma Related To Gravitational Challenge: Systematic Review Of Randomized Controlled Trials. It’s an article in the British Journal Of Medicine pointing out that there have never been any good studies proving that parachutes are helpful when jumping out of a plane, so they fail to meet the normal standards of evidence-based medicine. (...)
Of course this is a joke. It’s in the all-joke holiday edition of BMJ, and everyone involved knew exactly what they were doing. But the joke is funny because it points at something true. It’s biting social commentary. Doctors will not admit any treatment could possibly be good until it has a lot of randomized controlled trials behind it, common sense be damned. This didn’t come out of nowhere. They’ve been burned lots of times before by thinking they were applying common sense and getting things really wrong. And after your mistakes kill a few thousand people you start getting really paranoid and careful. And there are so many quacks who can spout off some “common sense” explanation for why their vitamin-infused bleach or colloidal silver should work that doctors have just become immune to that kind of bullshit. Multiple good RCTs or it didn’t happen. Given the history I think this is a defensible choice, and if you are tempted to condemn it you may find this story about bone marrow transplants enlightening.
But you can take this too far. After highlighting the lack of parachute RCTs, the paper continues:
Only two options exist. The first is that we accept that, under exceptional circumstances, common sense might be applied when considering the potential risks and benefits of interventions. The second is that we continue our quest for the holy grail of exclusively evidence based interventions and preclude parachute use outside the context of a properly conducted trial. The dependency we have created in our population may make recruitment of the unenlightened masses to such a trial difficult. If so, we feel assured that those who advocate evidence based medicine and criticise use of interventions that lack an evidence base will not hesitate to demonstrate their commitment by volunteering for a double blind, randomised, placebo controlled, crossover trial.
Did you follow that? For a good parachute RCT, half the subjects would have to jump out of a plane wearing a placebo parachute. The authors suggest maybe we enlist doctors who insist too stringently on RCTs over common sense for this dubious honor. (...)
But I would ask this of any journalist who pleads that they were just relaying and providing context for expert opinions: what was the experts’ percent confidence in their position?
I am so serious about this. What fact could possibly be more relevant? What context could it possibly be more important to give? I’m not saying you need to have put a number in your articles, maybe your readers don’t go for that. But were you working off of one? Did this question even occur to you?
Nate Silver said there was a 29% chance Trump would win. Most people interpreted that as “Trump probably won’t win” and got shocked when he did. What was the percent attached to your “coronavirus probably won’t be a disaster” prediction? Was it also 29%? 20%? 10%? Are you sure you want to go lower than 10%? Wuhan was already under total lockdown, they didn’t even have space to bury all the bodies, and you’re saying that there was less than 10% odds that it would be a problem anywhere else? (...)
People were presented with a new idea: a global pandemic might arise and change everything. They waited for proof. The proof didn’t arise, at least at first. I remember hearing people say things like “there’s no reason for panic, there are currently only ten cases in the US”. This should sound like “there’s no reason to panic, the asteroid heading for Earth is still several weeks away”. The only way I can make sense of it is through a mindset where you are not allowed to entertain an idea until you have proof of it. Nobody had incontrovertible evidence that coronavirus was going to be a disaster, so until someone does, you default to the null hypothesis that it won’t be.
by BMJ and Scott Alexander, Slate Star Codex | Read more:
[ed. A parody perhaps but one that highlights the problems of being overly concerned with not inducing a panic (or being tarred as an alarmist). Also, do check out the bone marrow transplant link for more relevance to our current situation (and the complexities of "weasels who rush in".]
In case you’ve been hiding under a rock recently (honestly, valid) the media not only failed to adequately warn its readers about the epidemic, but actively mocked and condescended to anyone who did sound a warning. Real Clear Politics has a list of highlights. The Vox tweet saying “Is this going to be a deadly pandemic? No.” Washington Post telling us in February “Why we should be wary of an aggressive government reponse to coronavirus (it might “scapegoat marginalized populations”). The Daily Beast complaining that “coronavirus, with zero American fatalities, is dominating headlines, while the flu is the real threat”. The New York Times, weighing in with articles like “The pandemic panic” and “Who says it’s not safe to travel to China”. The constant attempts to attribute “alarmism” over the virus to anti-Chinese racism. Etc, etc, etc.
One way people have summed this up is that the media (and the experts they relied on) did a terrible job predicting what would happen. I think this lets them off too easy. (...)
A few weeks ago, I wrote a blog post on face masks. It reviewed the evidence and found that they probably helped prevent the spread of disease. Then it asked: how did the WHO, CDC, etc get this so wrong?
I went into it thinking they’d lied to us, hoping to prevent hoarders from buying up so many masks that there weren’t enough for health workers. Turns out that’s not true. The CDC has been singing the same tune for the past ten years. Swine flu, don’t wear masks. SARS, don’t wear masks. They’ve been really consistent on this point. But why?
If you really want to understand what happened, don’t read any studies about face masks or pandemics. Read Smith & Pell (2003), Parachute Use To Prevent Death And Major Trauma Related To Gravitational Challenge: Systematic Review Of Randomized Controlled Trials. It’s an article in the British Journal Of Medicine pointing out that there have never been any good studies proving that parachutes are helpful when jumping out of a plane, so they fail to meet the normal standards of evidence-based medicine. (...)
Of course this is a joke. It’s in the all-joke holiday edition of BMJ, and everyone involved knew exactly what they were doing. But the joke is funny because it points at something true. It’s biting social commentary. Doctors will not admit any treatment could possibly be good until it has a lot of randomized controlled trials behind it, common sense be damned. This didn’t come out of nowhere. They’ve been burned lots of times before by thinking they were applying common sense and getting things really wrong. And after your mistakes kill a few thousand people you start getting really paranoid and careful. And there are so many quacks who can spout off some “common sense” explanation for why their vitamin-infused bleach or colloidal silver should work that doctors have just become immune to that kind of bullshit. Multiple good RCTs or it didn’t happen. Given the history I think this is a defensible choice, and if you are tempted to condemn it you may find this story about bone marrow transplants enlightening.
But you can take this too far. After highlighting the lack of parachute RCTs, the paper continues:
Only two options exist. The first is that we accept that, under exceptional circumstances, common sense might be applied when considering the potential risks and benefits of interventions. The second is that we continue our quest for the holy grail of exclusively evidence based interventions and preclude parachute use outside the context of a properly conducted trial. The dependency we have created in our population may make recruitment of the unenlightened masses to such a trial difficult. If so, we feel assured that those who advocate evidence based medicine and criticise use of interventions that lack an evidence base will not hesitate to demonstrate their commitment by volunteering for a double blind, randomised, placebo controlled, crossover trial.
Did you follow that? For a good parachute RCT, half the subjects would have to jump out of a plane wearing a placebo parachute. The authors suggest maybe we enlist doctors who insist too stringently on RCTs over common sense for this dubious honor. (...)
But I would ask this of any journalist who pleads that they were just relaying and providing context for expert opinions: what was the experts’ percent confidence in their position?
I am so serious about this. What fact could possibly be more relevant? What context could it possibly be more important to give? I’m not saying you need to have put a number in your articles, maybe your readers don’t go for that. But were you working off of one? Did this question even occur to you?
Nate Silver said there was a 29% chance Trump would win. Most people interpreted that as “Trump probably won’t win” and got shocked when he did. What was the percent attached to your “coronavirus probably won’t be a disaster” prediction? Was it also 29%? 20%? 10%? Are you sure you want to go lower than 10%? Wuhan was already under total lockdown, they didn’t even have space to bury all the bodies, and you’re saying that there was less than 10% odds that it would be a problem anywhere else? (...)
People were presented with a new idea: a global pandemic might arise and change everything. They waited for proof. The proof didn’t arise, at least at first. I remember hearing people say things like “there’s no reason for panic, there are currently only ten cases in the US”. This should sound like “there’s no reason to panic, the asteroid heading for Earth is still several weeks away”. The only way I can make sense of it is through a mindset where you are not allowed to entertain an idea until you have proof of it. Nobody had incontrovertible evidence that coronavirus was going to be a disaster, so until someone does, you default to the null hypothesis that it won’t be.
by BMJ and Scott Alexander, Slate Star Codex | Read more:
[ed. A parody perhaps but one that highlights the problems of being overly concerned with not inducing a panic (or being tarred as an alarmist). Also, do check out the bone marrow transplant link for more relevance to our current situation (and the complexities of "weasels who rush in".]
