3 ms·

The very next sentence after the famous "premature optimization is the root of all evil" says that "a 12% gain easily achieved should always be done". That sen

The very next sentence after the famous "premature optimization is the root of all evil" says that "a 12% gain easily achieved should always be done". That sentence was always taken out of context

bravura 3y ago
I know this is probably an unpopular position, but my ML/NLP Ph.D. advisor always said not to do any optimization unless it would yield 10x better performance. I guess that makes sense in the context of large-scale data research, where you should be thinking big if you really want to have outsized impact.
- alpaca128 3y ago
  > I guess that makes sense in the context of large-scale data research It makes the least sense in that context imho, as any optimization leads to larger differences the more data you process. Any performance difference is multiplied by the amount of times that code runs, that's why people look closer at hot loops than init code that runs once at startup.
- VHRanger 3y ago
  I mean that's obvious nonsense. Even in ML, which I work in, optimizing a data pipeline or model code to be 3x faster means you can run 3 experiments a day instead of 1. There's a huge chilling effect from slow code in research iteration. But par for the course as academic code tends to be absolute trash.
  - hedora 3y ago
    Ideally, a performance-related systems dissertation would achieve at least 10x vs. state of the art, but showing that is going to inevitably involve stacking many 10-20% optimizations just to get to par. Of course, there is plenty of low quality academic work that only compares to some existing system that has incomprehensibly bad performance, then shows a 10% improvement. That sort of research work should definitely be avoided.
    - VHRanger 3y ago
      My point isn't when you should be optimizing to the point that the optimization is publishable. My point is that you should be writing fast optimized code by default in ML research because you tend to throw large volumes of data at that code multiple times a day, and slow code reduces the number of experiments you can run. This isn't what I see however, lots of research code is horribly slow.
- usefulcat 3y ago
  Sometimes a position is unpopular for good reason.
- astrange 3y ago
  That's about when you should rewrite a project from scratch vs incrementally improving it, but of course it depends on the size of the project.

The very next sentence after the famous "premature optimization is the root of all evil" says that "a 12% gain easily achieved should always be done". That sentence was always taken out of context

bravura 3y ago

I know this is probably an unpopular position, but my ML/NLP Ph.D. advisor always said not to do any optimization unless it would yield 10x better performance. I guess that makes sense in the context of large-scale data research, where you should be thinking big if you really want to have outsized impact.

alpaca128 3y ago
> I guess that makes sense in the context of large-scale data research It makes the least sense in that context imho, as any optimization leads to larger differences the more data you process. Any performance difference is multiplied by the amount of times that code runs, that's why people look closer at hot loops than init code that runs once at startup.
VHRanger 3y ago
I mean that's obvious nonsense. Even in ML, which I work in, optimizing a data pipeline or model code to be 3x faster means you can run 3 experiments a day instead of 1. There's a huge chilling effect from slow code in research iteration. But par for the course as academic code tends to be absolute trash.
- hedora 3y ago
  Ideally, a performance-related systems dissertation would achieve at least 10x vs. state of the art, but showing that is going to inevitably involve stacking many 10-20% optimizations just to get to par. Of course, there is plenty of low quality academic work that only compares to some existing system that has incomprehensibly bad performance, then shows a 10% improvement. That sort of research work should definitely be avoided.
  - VHRanger 3y ago
    My point isn't when you should be optimizing to the point that the optimization is publishable. My point is that you should be writing fast optimized code by default in ML research because you tend to throw large volumes of data at that code multiple times a day, and slow code reduces the number of experiments you can run. This isn't what I see however, lots of research code is horribly slow.
usefulcat 3y ago
Sometimes a position is unpopular for good reason.
astrange 3y ago
That's about when you should rewrite a project from scratch vs incrementally improving it, but of course it depends on the size of the project.