3 ms·
Extending a trend beyond the interval it is measured is a big mistake. There are a lot of curves that fit the data in the low talen interval and the extend very
by gus_massa 4y ago
Extending a trend beyond the interval it is measured is a big mistake. There are a lot of curves that fit the data in the low talen interval and the extend very differently to the high talent interval.
If you don't want to get too fancy, a cubic polynomial will probably predict a fake huge advantage for teams with very high talent ratio.
I expect something like f(t)=C-expt(t/K) that looks like a quadratic for the low values but it's very flat near the high values.
I found many times good fits with f(t)=A log(t/B)+C.
In any case, the solution is to have a good model and amore data. Both are very complicated here, and extending the trend is a mistake, even as a unsupported conjeture.
I'm not sure if is should be retracted, but if you make a graphic like this in the physics lab, the T.A. will make you redo the whole presentation.
- hgsgm 4y agoThis was a big issue in Covid case forecasting: changing the model polynomial/formula and then fitting it to recent case count data points, gives different forecasts for when and how high the peak will be.
- ARandomerDude 4y ago[flagged]
- dang 4y agoPlease don't cross into unsubstantive comments or flamebait. "Comments should get more thoughtful and substantive, not less, as a topic gets more divisive." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- gus_massa 4y agoFor a short time prediction a quadratic/cubic is usually not that bad. I guess for covid-19 cases a week is fine. More than a week is probably too much, because the peaks last like a month from beginning to end, so it's too weird for a simple model. I guess you need a SIR model for that https://en.wikipedia.org/wiki/Compartmental_models_in_epidemiology https://en.wikipedia.org/wiki/Compartmental_models_in_epidem... Also, the grow phase is quite exponential, so even for a polynomial fit I'd prefer to use it after changing the vertical axis. Otherwise the polynomial must fit too many almost zeros and then jump, and you will get weird and negative results. There was also a strong weekly variation, cases on weekends were written down on Monday (or Tuesday), so I think it's better to look at the 7-day running average. The problem is that in this paper they are extending the "forecast" too much. It's somewhat the equivalent of 6 months. I thin no serious source tried to make very long term forecasts. (Anyway, some trends and general forecasts are possible, like with the flu.)
- apwheele 4y agoI will have to give these formulations a try. IMO pretty common to have asymptote like theoretical behavior in the tails with social science data, https://andrewpwheeler.com/2022/05/02/fitting-a-plateau-effects-model-in-scipy/ https://andrewpwheeler.com/2022/05/02/fitting-a-plateau-effe.... (Formulation is exactly quadratic for lower values and flattens at higher, and can re-parameterize to make it easier to fit.)