The Costly Failure to Update Sky-Is-Falling Predictions

Sean Trende for RealClearPolitics

On March 6, Liz Specht, Ph.D., posted a thread on Twitter that immediately went viral.  As of this writing, it has received over 100,000 likes and almost 41,000 retweets, and was republished at Stat News.  It purported to “talk math” and reflected the views of “highly esteemed epidemiologists.” It insisted it was “not a hypothetical, fear-mongering, worst-case scenario,” and that, while the predictions it contained might be wrong, they would not be “orders of magnitude wrong.” It was also catastrophically incorrect.

The crux of Dr. Specht’s 35-tweet thread was that the rapid doubling of COVID-19 cases would lead to about 1 million cases by May 5, 4 million by May 11, and so forth.  Under this scenario, with a 10% hospitalization rate, we would expect approximately 400,000 hospitalizations by mid-May, which would more than overwhelm the estimated 330,000 available hospital beds in the country.  This would combine with a lack of protective equipment for health care workers and lead to them “dropping from the workforce for weeks at a time,” to shortages of saline drips and so forth. Half the world would be infected by the summer, and we were implicitly advised to buy dry goods and to prepare not to leave the house.

Interestingly, this thread was wrong not because we managed to bend the curve and stave off the apocalypse; for starters, Dr. Specht described the cancellation of large events and workplace closures as something that would shift things by only days or weeks.

Instead, this thread was wrong because it dramatically overstated our knowledge of the way the virus worked; it fell prey to the problem, common among experts, of failing to address adequately the uncertainty surrounding its point estimates.  It did so in two opposing ways. First, it dramatically understated the rate of spread. If serological tests are to be remotely believed, we likely hit the apocalyptic milestone of 2 million cases quite some time ago.  Not in the United States, mind you, but in New York City, where 20% of residents showed positive COVID-19 antibodies on April 23.  Fourteen percent of state residents showed antibodies, suggesting 2.5 million cases in the Empire State alone; since antibodies take a while to develop, this was likely the state of affairs in mid-April or earlier.

But in addition to being wrong about the rate of spread, the thread was also very wrong about the rate of hospitalization. While New York City found its hospital system stretched, it avoided catastrophic failure, despite having within its borders the entire number of cases predicted for the country as a whole, a month earlier than predicted. Other areas of the United States found themselves with empty hospital beds and unused emergency capacity.

One would think that, given the amount of attention this was given in mainstream sources, there would be some sort of revisiting of the prediction. Of course, nothing of the sort occurred. This thread has been absolutely memory-holed, along with countless other threads and Medium articles from February and March. We might forgive such forays on sites like Twitter and Medium, but feeding frenzies from mainstream sources are also passed over without the media ever revisiting to see how things turned out. 

Consider Florida. Gov. Ron DeSantis was castigated for failing to close the beaches during spring break, and critics suggested that the state might be the next New York. I’ve written about this at length elsewhere, but Florida’s new cases peaked in early April, at which point it was a middling state in terms of infections per capita. The virus hasn’t gone away, of course, but the five-day rolling average of daily cases in Florida is roughly where it was in late March, notwithstanding the fact that testing has increased substantially. Taking increased testing into account, the positive test rate has gradually declined since late March as well, falling from a peak of 11.8% on April 1 to a low of 3.6% on May 12. 

Notwithstanding this, the Washington Post continues to press stories of public health officials begging state officials to close beaches (a more interesting angle at this point might be why these health officials were so wrong), while the New York Times noted a few days ago (misleadingly, and grossly so) that “Florida had a huge spike in cases around Miami after spring break revelry,” without providing the crucial context that the caseload mimicked increases in other states that did not play host to spring break. Again, perhaps the real story is that spring breakers passed COVID-19 among themselves and seeded it when they got home. I am sure some of this occurred, but it seems exceedingly unlikely that they would have spread it widely among themselves and not also spread it widely to bartenders, wait staff, hotel staff, and the like in Florida.

Florida was also one of the first states to experiment with reopening. Duval County (Jacksonville) reopened its beaches on April 19 to much national skepticism. Yet daily cases are lower today than they were they day that it reopened; there was a recent spike in cases associated with increased testing, but it is now receding.

Or consider Georgia, which one prominent national magazine claimed was engaging in “human sacrifice” by reopening. Yet, after nearly a month, a five-day average of Georgia’s daily cases looks like this:

What about Wisconsin, which was heavily criticized for holding in-person voting? It has had an increased caseload, but that is largely due to increased testing (up almost six-fold since early April) and an idiosyncratic outbreak in its meatpacking plants. The latter is tragic, but it is not related to the election; in fact, a Milwaukee Journal-Sentinel investigation failed to link any cases to the election; this has largely been ignored outside of conservative media sites such as National Review.  

We could go on – after being panned for refusing to issue a stay-at-home order, South Dakota indeed suffered an outbreak (once again, in its meatpacking plants), but deaths there have consistently averaged less than three per day, to little fanfare – but the point is made.  Some “feeding frenzies” have panned out, but many have failed to do so; rather than acknowledging this failure, the press typically moves on.

This is an unwelcome development, for a few reasons. First, not everyone follows this pandemic closely, and so a failure to follow up on how feeding frenzies end up means that many people likely don’t update their views as often as they should. You’d probably be forgiven if you suspected hundreds of cases and deaths followed the Wisconsin election.

Second, we obviously need to get policy right here, and to be sure, reporting bad news is important for producing informed public opinion. But reporting good news is equally as important. Third, there are dangers to forecasting with incredible certitude, especially with a virus that was detected less than six months ago. There really is a lot we still don’t know, and people should be reminded of this. Finally, among people who do remember things like this, a failure to acknowledge errors foments cynicism and further distrust of experts. The damage done to this trust is dangerous, for at this time we desperately need quality expert opinions and news reporting that we can rely upon.