3 ms·
Yes. I am worried that over the long term, models which are explainable or justifiable will be pushed aside by models which are nonexplainable and nonjustifiabl
by klodolph 4y ago
Yes. I am worried that over the long term, models which are explainable or justifiable will be pushed aside by models which are nonexplainable and nonjustifiable, because the nonexplainable / nonjustifiable models are cheaper to get running and in the end we are all trapped inside a system that ruthlessly optimizes short-term costs.
I've only done a small amount of statistical work professionally, but I remember that it was basically a nightmare to try and collect the right data and use the right variables to get a useful inference. Other engineers on my team would try to improve the models by throwing more variables at them or increasing the number of parameters in the model and I was always pushing back on this. I felt a bit crazy pushing back on this--surely, using more variables and more parameters will make the model fit historical data more closely, and a model which fits historical data more closely will fit future data more closely, and a model which fits future data more closely will be more useful?
Explaining why this is wrong is really hard. I have deep respect for people who are able to explain the principles of statistics and statistical modeling.
- thaumasiotes 4y ago> I felt a bit crazy pushing back on this--surely, using more variables and more parameters will make the model fit historical data more closely, and a model which fits historical data more closely will fit future data more closely, and a model which fits future data more closely will be more useful? > Explaining why this is wrong is really hard. Seems straightforward. Points 1 and 3 are correct; point 2 is false.
- klodolph 4y agoPoints 1 and 2 are both wrong also. Statistics is not intuitive.
- thaumasiotes 4y agoPoint 1 isn't wrong; it's correct. There's a reason the name for this problem is "overfitting".
- klodolph 4y agoYeah, it sounds like you’re assuming two things things here which are often (or usually) wrong: 1. No cross-validation, and 2. You have the ability to solve for additional parameters without losing accuracy. Statistics is not intuitive.