3 ms·
It's even worse than that. Talking from personal experience in the private sector: You go trough the whole process and present your results for a customer.
by nabla9 10y ago
It's even worse than that.
Talking from personal experience in the private sector:
You go trough the whole process and present your results for a customer. They make the required changes, see few percent improvement in the bottom line and are happy. Year later you pick up the same code for another project and discover small error in data collection script that completely invalidates everything.
Garbage in, garbage out errors can have internal consistency that survive cross validation and provide similar results with many different algorithms and models. Random changes in the real word can produce actual gains.
Standard machine learning datasets rarely suffer from this problem because results and semantics of the problem are known beforehand. If you have a original black-box problem, it's possible to do random search and improve by accident.