I met a dude once who was using LSTMs to predict HVAC failures for some sort of HVAC support firm. On some interrogation, it turned out that the input data was

I met a dude once who was using LSTMs to predict HVAC failures for some sort of HVAC support firm. On some interrogation, it turned out that the input data was garbage, and the only way the model "worked" is if it was repeatedly overfit on only the last week's worth of data... comprising about 5k rows, iirc. This struck me as completely terrifying... it's not like the set of HVAC units was changing out every week. And one would hope that the historical data would help arrive at a general solution. My guess is they had some sort of church-meets-state problem in the test/train split, so that the short-term data allowed 'predicting' the test set. Dude was quite convinced that they were making HVAC history, though.