So we do the loop 50 time and we now have an algorithm that works (97%!) on the test set. We are happy! We run it in production and everything looks good (prbly

So we do the loop 50 time and we now have an algorithm that works (97%!) on the test set. We are happy! We run it in production and everything looks good (prbly 92% ish). Everyone is happy! We all get promoted or get new jobs. Then, one day, someone actually looks at what it's doing... and lo. It. does. not. work (~51%) Everyone is sad. Apart from us! Yay! Seriously - an optimisation loop on a test set? Seriously?