Oh no: > I have done enough research on production ML now to know that it pays to simply overfit to the most recent data and constantly retrain. Successful com

Oh no: > I have done enough research on production ML now to know that it pays to simply overfit to the most recent data and constantly retrain. Successful companies do this. We all know that overfitting is bad (e.g. in time-series forecasting, the past isn't always representative of the future). Depending on your domain, more recent data may be more valuable than older data, sure. The solution is not to overfit to recent data! In my experience, it is to design features which take into account recency. For example, in a particular quantity we wanted to forecast, we found out that using ~7 days worth of data was better than using multiple months, due to the data being non-stationary (the mean of the quantity was changing over time). What we did was combine features with an exponential decay with the appropriate decay constant, to great results.