COVID projections not taking schools etc. opening into consideration?
September 13, 2020 1:02 PM Subscribe
I'm looking at Covid infection rate projections for New York City, such as this one. As you can see, the line is quite flat until November 1st, which is as far as they are projecting into the future.
This seems strange to me, with all the brouhaha about opening schools and various businesses. Some colleges are already open (although many students are staying remote).
I've looked at a number of these projection sites, and they seem to have gotten the numbers right so far, but could it really be that, given schools, restaurants, and other businesses opening up, plus the weather getting colder and flu season perhaps beginning, there will be no major increases in infections/cases until after November 1st? (Obviously, I would like to think that this is the case, but it doesn't seem logical to me.)
Also, if you know of any sites that project beyond December 1st, I'd like to see them. I haven't been able to find any myself. Thanks.
This seems strange to me, with all the brouhaha about opening schools and various businesses. Some colleges are already open (although many students are staying remote).
I've looked at a number of these projection sites, and they seem to have gotten the numbers right so far, but could it really be that, given schools, restaurants, and other businesses opening up, plus the weather getting colder and flu season perhaps beginning, there will be no major increases in infections/cases until after November 1st? (Obviously, I would like to think that this is the case, but it doesn't seem logical to me.)
Also, if you know of any sites that project beyond December 1st, I'd like to see them. I haven't been able to find any myself. Thanks.
Disclaimer: not an expert, not a data scientist or an immunologist. I would assume that the flat lines are more likely to be due to the data needing to be projected forward a certain amount rather than being any indicator of what's likely to happen.
Just to take my own city (London, UK) as an example, projections from last month might have put us in a similar position, but right now we're looking at an increased rate of exponential growth after a couple of months of reopenings (though schools and universities haven't been back long enough to have much of an impact yet).
In any case, I would take these sorts of graphs as a vague assumption at best, somewhere around the area of hanging fish on a line outside to predict the weather. To be honest, I would put anything predicting beyond a week in that range, right now.
posted by fight or flight at 1:49 PM on September 13, 2020 [2 favorites]
Just to take my own city (London, UK) as an example, projections from last month might have put us in a similar position, but right now we're looking at an increased rate of exponential growth after a couple of months of reopenings (though schools and universities haven't been back long enough to have much of an impact yet).
In any case, I would take these sorts of graphs as a vague assumption at best, somewhere around the area of hanging fish on a line outside to predict the weather. To be honest, I would put anything predicting beyond a week in that range, right now.
posted by fight or flight at 1:49 PM on September 13, 2020 [2 favorites]
To quote the forecaster you linked to, "We don't have projections beyond Nov 1 because we want to avoid misleading headlines (like below). No one knows what's going to happen in 2+ months. Forecasting certainty when there is very little can undermine public trust in the scientific community." Any prediction you see is nothing more than a slightly informed way to draw a line through points on a graph. Please don't be fooled by the use of sophisticated statistical techniques; no one knows anything. Truly. Everyone, ferociously smart statisticians included, desperately wants to know something, but they don't. If you read Gu's discussion they are saying, in a precise and technical way, that they have very little confidence and are making a ton of assumptions in order to produce their results. If any forecaster says otherwise, they are lying.
If you want to see how much uncertainty, I would consult the FiveThirtyEight forecast roundup. The models don't even agree to within 10K people how many will die between now and three weeks from now. You can go back in time by changing the date from "See forecasts from Today" to sometime in the past and see how they did. Most of the models do not hit the actual number exactly, though they do typically include it in their 95% confidence band. But that's a low bar. If we were predicting coin flips, getting heads 4 times in a row would be inside the 95% band, even though we can all agree that's darn unlikely.
posted by wnissen at 2:32 PM on September 13, 2020 [2 favorites]
If you want to see how much uncertainty, I would consult the FiveThirtyEight forecast roundup. The models don't even agree to within 10K people how many will die between now and three weeks from now. You can go back in time by changing the date from "See forecasts from Today" to sometime in the past and see how they did. Most of the models do not hit the actual number exactly, though they do typically include it in their 95% confidence band. But that's a low bar. If we were predicting coin flips, getting heads 4 times in a row would be inside the 95% band, even though we can all agree that's darn unlikely.
posted by wnissen at 2:32 PM on September 13, 2020 [2 favorites]
I'm not entirely sure what the question in your title is, there's a question mark at the end but that's mot actually a question.
On the page you linked, there's a link to their assumptions -- these are the things they considered in making projections. It will tell you in detail what they considered.
https://covid19-projections.com/about/#second-wave
Projections into the future at this point are basically an imformed guess. The further in the future, the less informed the guess, so that's why you aren't finding projections further out -- no one knows. And if someone is certain they know, they aren't likely to be a reliable source.
posted by yohko at 4:44 PM on September 13, 2020 [1 favorite]
On the page you linked, there's a link to their assumptions -- these are the things they considered in making projections. It will tell you in detail what they considered.
https://covid19-projections.com/about/#second-wave
Projections into the future at this point are basically an imformed guess. The further in the future, the less informed the guess, so that's why you aren't finding projections further out -- no one knows. And if someone is certain they know, they aren't likely to be a reliable source.
posted by yohko at 4:44 PM on September 13, 2020 [1 favorite]
I am so far from being an epidemiologist that one of them would need a telescope to see me.
But.
The underlying model they're using is a very simple model using, it appears to my utterly untrained eye, a few differential equations. The underlying model doesn't know about schools that are going to open, it doesn't know about people not wearing masks, it doesn't know that it's going to get cold. It doesn't care about any of that. What it knows is how many people died on what day in what place. The machine learning part of it tweaks those differential equations to get more or less the best fit to the existing data about deaths, and then runs that specific set of differential equations out a bit into the future.
If deaths start going up because of schools or cold weather, the model will re-tweak the differential equations to include that uptick, and run that *new* set of diffeqs out into the future. But it seems to have no capacity whatsoever to predict how covid rates will react to changing circumstances.
posted by GCU Sweet and Full of Grace at 6:22 PM on September 13, 2020
But.
The underlying model they're using is a very simple model using, it appears to my utterly untrained eye, a few differential equations. The underlying model doesn't know about schools that are going to open, it doesn't know about people not wearing masks, it doesn't know that it's going to get cold. It doesn't care about any of that. What it knows is how many people died on what day in what place. The machine learning part of it tweaks those differential equations to get more or less the best fit to the existing data about deaths, and then runs that specific set of differential equations out a bit into the future.
If deaths start going up because of schools or cold weather, the model will re-tweak the differential equations to include that uptick, and run that *new* set of diffeqs out into the future. But it seems to have no capacity whatsoever to predict how covid rates will react to changing circumstances.
posted by GCU Sweet and Full of Grace at 6:22 PM on September 13, 2020
(Sorry, that's inaccurate; their baseline model that they start the machine learning on has some of those things baked in. I do question how much remains after the machine learning is done.)
posted by GCU Sweet and Full of Grace at 6:26 PM on September 13, 2020
posted by GCU Sweet and Full of Grace at 6:26 PM on September 13, 2020
I wouldn't trust them more than a couple of weeks out. Deaths can probably be reasonably accurately predicted from current case numbers (if ages are reported with that data).
Both NSW and VIC (similar populations) in Australia had about 5-10 new cases per day at the beginning of June. A month later, Victoria was in it's second wave, 10s of cases per day and locking down again, by August it was 100s of cases per day. NSW popped upto about 20 cases per day for awhile, but seems to have stabilised again.
To answer your question about schools, the limitations page says that they have not included that information in their models.
posted by kjs4 at 9:05 PM on September 13, 2020 [1 favorite]
Both NSW and VIC (similar populations) in Australia had about 5-10 new cases per day at the beginning of June. A month later, Victoria was in it's second wave, 10s of cases per day and locking down again, by August it was 100s of cases per day. NSW popped upto about 20 cases per day for awhile, but seems to have stabilised again.
To answer your question about schools, the limitations page says that they have not included that information in their models.
posted by kjs4 at 9:05 PM on September 13, 2020 [1 favorite]
Specific to New York City, in case helpful, schools there don't reopen for another week.
posted by troywestfield at 6:52 AM on September 14, 2020 [1 favorite]
posted by troywestfield at 6:52 AM on September 14, 2020 [1 favorite]
« Older How can I find someone to help with packing? | Help me spice up my (distanced) sex life Newer »
This thread is closed to new comments.
posted by DMelanogaster at 1:03 PM on September 13, 2020