10 ms·

It should be quite easy to see the value of optimizing compilers. Compile your program with optimizations turned off. Now make it as fast as your release build

It should be quite easy to see the value of optimizing compilers. Compile your program with optimizations turned off. Now make it as fast as your release build again, while still keeping them off. For much of my code, I think this would take years.

nullc 9y ago
You may be significantly underestimating your ability to get phenomenal gains from algorithmic improvements. I think DJB would argue that the compiler optimizations are insignificant in comparison to those improvements for a very large portion of hot code. Of course, even if I'm correct you could turn the optimizations back on and be faster still. One reason why you might not want to do that is the cost of increased miscompliation.
- DannyBee 9y ago
  " I think DJB would argue that the compiler optimizations are insignificant in comparison to those improvements for a very large portion of hot code." Except he has no data to show this, and every piece of data i have (and my colleagues have), says that he is wrong. Additionally, the compilers can and do replace algorithms if users let them. Most users don't want it, even if it makes code faster. Remember that the vast majority of people want software that works and is reliable, or has fast development cycles, or a million other things, and happily choose that over software that is fast. Compilers changing algorithms on you isn't highly compatible with this, especially when, for example, most programming languages don't even give you a good way to express the invariants you are trying to maintain (except good ol' Ada). However, besides that, certainly you can see that in the end, the people lose. It's like arguing that computer would never win at go. If i really really really cared about some piece of hot code, i wouldn't pay an expert to optimize it, i'd throw the cycles of 100k spare cpus at optimizing it. That is also the other strange part of his argument to me, he assumes the capabilities of compilers based on compilers he sees that are meant to operate pretty much single threaded on single machines, and complains "they will never beat people". It's like looking at the machine learning algorithm that takes months or years or roughly forever to train on a single cpu and say "it'll never do a good job". The world is not that small anymore. It's blindingly obvious the optimizing compilers win if we want them to. In any case, this particular debate will restart again as gpus and accelerators follow the same old compiler cycle.
  - oconnor0 9y ago
    > Additionally, the compilers can and do replace algorithms if users let them. Most users don't want it, even if it makes code faster. What compilers do this? With what algorithms?
    - jerven 9y ago
      If you look at modern JIT compilers such as Truffle-Ruby, then yes it will change algorithms. e.g. changing sort algorithms depending on data size. if the data is small enough it will do a bubblesort on registers, when its big it will do TimSort on heap data etc...[1] [1]: http://chrisseaton.com/rubytruffle/small-data-structures/ http://chrisseaton.com/rubytruffle/small-data-structures/
      - coldtea 9y ago
        This example is about the compiler changing between (existing) internal versions of sorting algorithms for language data structures. Not about changing what algorithms the user coded.
        danielbarla 9y ago
        Exactly, that's two completely different things. It's akin to a compiler using the famous "switch statement becomes if-else chain if less than 7 items, otherwise lookup table". The parent posts are referring to macro-level optimisations, where large changes to how a system works are implemented, in order to get massive gains in performance. Picking out the essence of the larger implementation would be a difficult task for a compiler, but more importantly it wouldn't have enough information about the context of the application to make optimisation decisions. Things like: do we need to read from that file each time we need this piece information, or is it a once-off that can be cached? Unless this contextual information is expressed somehow (and it basically never is), only the programmer will be able to make the change.
        jerven 9y ago
- oconnor0 9y ago
  I think one of the issues here is what compiler optimizations encompass. In a language like C or C++, you may be able to write optimal algorithms but end up with poor performance because of memory access. An unoptimized compile puts almost everything on the stack - a = b + c involves two loads, one add, and one store in an unoptimized compile. All of that extra memory access is going to kill performance; even if you have optimal data structures and algorithms. I see "optimizations are bad/unnecessary/problematic/whatever" in my job writing a compiler. The underlying issue is almost never "I don't want optimizations" - even when someone thinks that's what they want; it's "I don't want optimizations that produce unexpected behavior".
- dagw 9y ago
  You may be significantly underestimating your ability to get phenomenal gains from algorithmic improvements. My HPC lecturer used to say that the speed difference between bad Matlab code and good Matlab code will often be greater than the speed difference between good Matlab code and good Fortran code. Tweaking the hell out of your code at the assembly level is a waste of time if you're using a naive O(2^N) algorithm.
  - gpderetta 9y ago
    ... unless N is small.
  - sllabres 9y ago
    You mean O(n^2). 2^n looks a more than 'naive' for most cases.
- dom0 9y ago
  > You may be significantly underestimating your ability to get phenomenal gains from algorithmic improvements. Implementation differences in the exact same, say, simple linear algorithm, can still add up to a tenfold difference. Compiler optimisations can take it further, though some are not generally applicable. E.g. I have made good experiences with PGO (careful analysis and/or knowing your workload is needed), but it's obviously not an option for the typical open source software distribution model. Obviously, combining a better algorithm with a better implementation that is optimized will be fastest.
- Sean1708 9y ago
  > You may be significantly underestimating your ability to get phenomenal gains from algorithmic improvements. Absolutely but often it's very hard to make those changes or those changes lead to a loss of accuracy, whereas compiling with -O3 is very easy and leads to no loss in accuracy (in the common case).
std_throwaway 9y ago
People use Python to actually do stuff. Performance-wise it's a nightmare but usability outweighs it in many many cases.
- coldtea 9y ago
  Plus it's only a nightmare "performance-wise" when it's actually slow in wall clock time. Often it's so fast that it doesn't matter at all that it can be 1/100 slower then C. 0.1 sec is still good even if it's not 0.001 secs.
  - dom0 9y ago
    Python performance is uh ah... difficult. I work on a piece of software that has a lot of hot paths in Python code, and it's not nice. Combined with deployment and tooling issues it made me wonder quite a few times now, whether the a little bit lower contribution barrier is worth all these hassles.
    - aeroevan 9y ago
      Cython is a very nice way to transition hot python paths into a C extension. The best part is that it's a python superset, so it should provide some speed up without any typing information (and can be very fast once the hot paths are all in cdef'd functions).
      - nickpsecurity 9y ago
        Beat me to it. Cython was a great solution to this problem.
      - dom0 9y ago
        https://news.ycombinator.com/item?id=14376961 https://news.ycombinator.com/item?id=14376961 Further, it doesn't help with the required restructuring; when the functionality is scattered across a dozen or more modules (complex software + good Python practice), then you first have to pull the functionality needed for the path you're trying to get fast together. That leads to duplication, removing abstractions and most certainly vetoes from other people.
wolfgke 9y ago
> It should be quite easy to see the value of optimizing compilers. Compile your program with optimizations turned off. Now make it as fast as your release build again, while still keeping them off. For much of my code, I think this would take years. The problem is that, say, C does not allow to express the necessary details about the optimizations that you want to do. This is exactly something DJB calls for in programming languages.

nullc 9y ago

You may be significantly underestimating your ability to get phenomenal gains from algorithmic improvements. I think DJB would argue that the compiler optimizations are insignificant in comparison to those improvements for a very large portion of hot code. Of course, even if I'm correct you could turn the optimizations back on and be faster still. One reason why you might not want to do that is the cost of increased miscompliation.

DannyBee 9y ago
" I think DJB would argue that the compiler optimizations are insignificant in comparison to those improvements for a very large portion of hot code." Except he has no data to show this, and every piece of data i have (and my colleagues have), says that he is wrong. Additionally, the compilers can and do replace algorithms if users let them. Most users don't want it, even if it makes code faster. Remember that the vast majority of people want software that works and is reliable, or has fast development cycles, or a million other things, and happily choose that over software that is fast. Compilers changing algorithms on you isn't highly compatible with this, especially when, for example, most programming languages don't even give you a good way to express the invariants you are trying to maintain (except good ol' Ada). However, besides that, certainly you can see that in the end, the people lose. It's like arguing that computer would never win at go. If i really really really cared about some piece of hot code, i wouldn't pay an expert to optimize it, i'd throw the cycles of 100k spare cpus at optimizing it. That is also the other strange part of his argument to me, he assumes the capabilities of compilers based on compilers he sees that are meant to operate pretty much single threaded on single machines, and complains "they will never beat people". It's like looking at the machine learning algorithm that takes months or years or roughly forever to train on a single cpu and say "it'll never do a good job". The world is not that small anymore. It's blindingly obvious the optimizing compilers win if we want them to. In any case, this particular debate will restart again as gpus and accelerators follow the same old compiler cycle.
- oconnor0 9y ago
  > Additionally, the compilers can and do replace algorithms if users let them. Most users don't want it, even if it makes code faster. What compilers do this? With what algorithms?
  - jerven 9y ago
    If you look at modern JIT compilers such as Truffle-Ruby, then yes it will change algorithms. e.g. changing sort algorithms depending on data size. if the data is small enough it will do a bubblesort on registers, when its big it will do TimSort on heap data etc...[1] [1]: http://chrisseaton.com/rubytruffle/small-data-structures/ http://chrisseaton.com/rubytruffle/small-data-structures/
    - coldtea 9y ago
      This example is about the compiler changing between (existing) internal versions of sorting algorithms for language data structures. Not about changing what algorithms the user coded.
      - danielbarla 9y ago
        Exactly, that's two completely different things. It's akin to a compiler using the famous "switch statement becomes if-else chain if less than 7 items, otherwise lookup table". The parent posts are referring to macro-level optimisations, where large changes to how a system works are implemented, in order to get massive gains in performance. Picking out the essence of the larger implementation would be a difficult task for a compiler, but more importantly it wouldn't have enough information about the context of the application to make optimisation decisions. Things like: do we need to read from that file each time we need this piece information, or is it a once-off that can be cached? Unless this contextual information is expressed somehow (and it basically never is), only the programmer will be able to make the change.
        jerven 9y ago
oconnor0 9y ago
I think one of the issues here is what compiler optimizations encompass. In a language like C or C++, you may be able to write optimal algorithms but end up with poor performance because of memory access. An unoptimized compile puts almost everything on the stack - a = b + c involves two loads, one add, and one store in an unoptimized compile. All of that extra memory access is going to kill performance; even if you have optimal data structures and algorithms. I see "optimizations are bad/unnecessary/problematic/whatever" in my job writing a compiler. The underlying issue is almost never "I don't want optimizations" - even when someone thinks that's what they want; it's "I don't want optimizations that produce unexpected behavior".
dagw 9y ago
You may be significantly underestimating your ability to get phenomenal gains from algorithmic improvements. My HPC lecturer used to say that the speed difference between bad Matlab code and good Matlab code will often be greater than the speed difference between good Matlab code and good Fortran code. Tweaking the hell out of your code at the assembly level is a waste of time if you're using a naive O(2^N) algorithm.
- gpderetta 9y ago
  ... unless N is small.
- sllabres 9y ago
  You mean O(n^2). 2^n looks a more than 'naive' for most cases.
dom0 9y ago
> You may be significantly underestimating your ability to get phenomenal gains from algorithmic improvements. Implementation differences in the exact same, say, simple linear algorithm, can still add up to a tenfold difference. Compiler optimisations can take it further, though some are not generally applicable. E.g. I have made good experiences with PGO (careful analysis and/or knowing your workload is needed), but it's obviously not an option for the typical open source software distribution model. Obviously, combining a better algorithm with a better implementation that is optimized will be fastest.
Sean1708 9y ago
> You may be significantly underestimating your ability to get phenomenal gains from algorithmic improvements. Absolutely but often it's very hard to make those changes or those changes lead to a loss of accuracy, whereas compiling with -O3 is very easy and leads to no loss in accuracy (in the common case).

std_throwaway 9y ago

People use Python to actually do stuff. Performance-wise it's a nightmare but usability outweighs it in many many cases.

coldtea 9y ago
Plus it's only a nightmare "performance-wise" when it's actually slow in wall clock time. Often it's so fast that it doesn't matter at all that it can be 1/100 slower then C. 0.1 sec is still good even if it's not 0.001 secs.
- dom0 9y ago
  Python performance is uh ah... difficult. I work on a piece of software that has a lot of hot paths in Python code, and it's not nice. Combined with deployment and tooling issues it made me wonder quite a few times now, whether the a little bit lower contribution barrier is worth all these hassles.
  - aeroevan 9y ago
    Cython is a very nice way to transition hot python paths into a C extension. The best part is that it's a python superset, so it should provide some speed up without any typing information (and can be very fast once the hot paths are all in cdef'd functions).
    - nickpsecurity 9y ago
      Beat me to it. Cython was a great solution to this problem.
    - dom0 9y ago
      https://news.ycombinator.com/item?id=14376961 https://news.ycombinator.com/item?id=14376961 Further, it doesn't help with the required restructuring; when the functionality is scattered across a dozen or more modules (complex software + good Python practice), then you first have to pull the functionality needed for the path you're trying to get fast together. That leads to duplication, removing abstractions and most certainly vetoes from other people.

wolfgke 9y ago

> It should be quite easy to see the value of optimizing compilers. Compile your program with optimizations turned off. Now make it as fast as your release build again, while still keeping them off. For much of my code, I think this would take years. The problem is that, say, C does not allow to express the necessary details about the optimizations that you want to do. This is exactly something DJB calls for in programming languages.