Great opportunity to talk about time series predictions! It's not obvious when you're starting out with this type of prediction how you should measure accuracy.

Great opportunity to talk about time series predictions! It's not obvious when you're starting out with this type of prediction how you should measure accuracy. The simplest approach is to compare the guess made for a point to the actual data as was done here. "People have over- and underguesstimated the number of new cases, but not by much. There were only a few occasions where the actual case count didn’t fall within one standard deviation of the mean of guesses (represented by the whiskers around blue bars on the plot). Granted, the daily standard deviation tends to be large (on the order of a few thousand), but still, I’m impressed." The author almost calls this out but there is a natural range that the data is expected to fall into given the previous data. (Standard deviation across the whole time series isn't actually that helpful). This leads us to what should be everyone's first baseline. Last-time-step prediction. If your prediction is simply the previous true value how good are your predictions? The mean absolute error over some time window is a decent place to start when you're measuring quality. This predict-the-last-timestep model is often surprisingly good. If your prediction horizon is really only one timestep then its easy to fool yourself that your model is doing well. This is why models are usually judged on how quickly the diverge from the true data over as long a time horizon as possible. If your model is capturing underlying dynamics then it should be better at predicting long time horizons. There are, of course, many ways to then start building up better and better models for use as baselines, and even relatively simple models (random forest over a moving window for example) can produce pretty good predictors. I'd love to see how humans do in compared to those over medium to long term predictors of covid case counts!