Yeah, the training data is basically useless: 1. Extreme imbalance and survivorship bias. The dataset is unwieldy because of the imbalance, no matter the algor

Yeah, the training data is basically useless: 1. Extreme imbalance and survivorship bias. The dataset is unwieldy because of the imbalance, no matter the algorithm; and too small if you balance it. 2. I would take a daily/weekly/monthly diff between different values and time to failure, and obviously only from drives that ACTUALLY failed. All drives fail, but data from healhty drives is useless as it lacks the target value. In practice you want to predict how close they are to failing and act preemptively, tagging them as OK and KO completely misses the target (hehe) IMO. 3. Drive maker and model should be vectorized and fed into the model. Mandatory. Different manufacturers and models show different SMART behaviors, and while some stats are red flags in certain drives, they mean nothing in others. 4. This is more of a personal preference, but I like random trees to test whether the dataset is useful. They're not perfect but they give reasonably good results for most tasks, and tend not to overfit, so you can use it as overfitting benchmark vs other algorithms. If they don't either the dataset is shit or it needs some transformation, or the model needs to be more complex (but that's easy to see). Obviously this applies to classifiers and regressions, and not to many other ML tasks.