My point isn't when you should be optimizing to the point that the optimization is publishable. My point is that you should be writing fast optimized code by d

My point isn't when you should be optimizing to the point that the optimization is publishable. My point is that you should be writing fast optimized code by default in ML research because you tend to throw large volumes of data at that code multiple times a day, and slow code reduces the number of experiments you can run. This isn't what I see however, lots of research code is horribly slow.