Great insight here that a lot of performance drift is down to data and engineering, not "natural" non-stationarity. The corollary is that one of the hidden str

Great insight here that a lot of performance drift is down to data and engineering, not "natural" non-stationarity. The corollary is that one of the hidden strengths of ML systems is the ability to adapt to imperfect inputs, and even changing inputs with retraining. Re: monitoring -- When I was doing automated trading, we of course had automated alerts to stop systems that were outside expected parameters. However, as OP describes, there was also a long tail of degradations where there was no great precision/recall tradeoff on flagging "real" issues. Instead, we put effort into "calibration reports" that visually surfaced as much information about training and performance as our eyes could handle. These would include things like over time plots and PCA plots for recent feature histories, where it was more an opportunity to spot patterns than an explicit metric. Reviewing these for 15m each day with our own eyes was much more effective at detecting a long tail of unanticipated degradations than we would have been at anticipating and coding for each one explicitly -- and it help build intuition that fueled new research ideas.