4 ms·

I recently wrote a simulation that involved some heavy numerical calculations. Without optimization it was dog slow and would have taken probably years to produ

I recently wrote a simulation that involved some heavy numerical calculations. Without optimization it was dog slow and would have taken probably years to produce the results I needed. With compiler optimization it took a few days, which was good enough for me. Perhaps I could have improved on that with manual optimizations, but the speed was good enough so instead of spending time optimizing I did other things. The implicit assumption in the talk (based on this summary) is that we always want the most optimized code. But in fact that is never the case -- what we really want is for the code to be fast enough (or be space-optimized enough). So the relevant question is this: in what percentage of cases do compiler optimizations take us from 'not fast enough' to 'fast enough'. In all these cases the compiler is useful, because it saves us the cost of doing our own optimizations. In my experience this percentage is far from negligible, so at least for me optimizing compilers are highly valuable.

panic 11y ago
I think there's an interesting distinction here between general-purpose software and single-use code. General-purpose software has many users with many kinds of data. Unless your domain is intrinsically limited, it's impossible to be "fast enough" -- you can always find some data that your code will take to long to process. There's always an incremental benefit to optimization, and with enough users, it's irresponsible to waste collective years of people's time by leaving performance on the table. What you've said makes perfect sense for one-off simulations, however. And in practice, optimizers are designed to make tight numerical code run fast. I don't think your use case will ever be ignored. It's just that most systems today are not made of tight loops of arithmetic operations. We need more tools to optimize the kind of code that actually makes our computers slow: code that allocates and deallocates memory, code that transforms data between different formats, code that constructs and operates on complex data structures -- that sort of thing.
- valm- 11y ago
  I don't entirely disagree, but I would like to make one point. Taking 1 millisecond from 1000 people, does not translate to 1 second of lost human productivity. There's no way to collect all those little slices of time and they are not appreciable individually. (Assuming those milliseconds are sufficiently spread out, i.e., not 1 ms out of every 2, or something like that)
  - modeless 11y ago
    > Taking 1 millisecond from 1000 people, does not translate to 1 second of lost human productivity I believe that it does. If you average over a large population even tiny imperceptible increases in e.g. page load time have a measurable effect. Perhaps 1000 people isn't a large enough sample to measure the effect from 1ms delay considering all the confounding factors at play, but I absolutely believe you could see it with precise enough measurement and a large enough sample.
    - Rebelgecko 11y ago
      Reminds me of this semi-relevant story about improving the boot times of the original Macintosh — http://www.folklore.org/StoryView.py?story=Saving_Lives.txt http://www.folklore.org/StoryView.py?story=Saving_Lives.txt
      - gaius 11y ago
        The irony being that Jobs himself limited the memory in the original Mac, so it would have to go to disk more often!
        jedrek 11y ago
        Source?
        gaius 11y ago
        Its a fairly well known story. The engineers ignored him and built in capability for 512k anyway. http://www.informationweek.com/mobile/mobile-devices/steve-jobs-jerry-pournelle-reflects/d/d-id/1099842 http://www.informationweek.com/mobile/mobile-devices/steve-j... He also hated fans and expansion slots.
    - darkmighty 11y ago
      Hmm I think the heart of the question is whether productivity is linear with time. I guess he's right that, for small values of t, productivity is highly superlinear. So giving 1 person an extra minute has an impact on that person, but because of superlinearity, if you distribute that among 60 persons the extra second won't allow anything new, like an original thought or whatever.
- chriswarbo 11y ago
  > There's always an incremental benefit to optimization, and with enough users, it's irresponsible to waste collective years of people's time by leaving performance on the table. I could just as easily claim that skilled developer's time is a precious resource, and it's irresponsible to waste it squeezing out performance when so much software still remains unwritten.
  - bostik 11y ago
    There's another side to that, as well. Developers' time is expensive, and the time it takes to get feedback from code runs adds up when you have lots of developers. Especially if the roundtrip times are short enough not to allow switching between tasks, but long enough to cause constant "wall time waits". I'd be more than willing to have couple of extremely skilled developers profile and optimise the common code so that the feedback loop becomes shorter for everyone. That way more of the yet-unwritten software might just be tackled. My old boss once had me spend more than a week packaging and streamlining our access VPN configurations. When I asked him about it some time later, he explained that even though I was away from billable projects for that time, it would have cost a whole lot more overall to have all our developers and managers massage their setups every time something changed. Large figures are sums of lots of small figures...
gaius 11y ago
A typical compiler takes -O -O2 and so on flags for how much optimization you want. But what I would really like is a -O60s meaning, do as much optimization as you can in 60s and we're good to go. For the production version I might give it -O1h or -O12h and run it overnight. I am not aware of any product that does this.
- TheLoneWolfling 11y ago
  The one issue with this type of compilation is that it encourages hisenbugs. You add debugging code, it can't spend as much time optimizing, and all of a sudden it is producing the correct results again. But none the less, I agree.
tripzilch 11y ago
I agree it's a trade-off, it always is :) But if the goal is to write performant code, there's a few more variables to consider. An important one, IMO, is experience and practice. Optimization can be found in the weirdest places. My latest experience is not with C or assembly instruction crunching, but with Processing/Java. If I weren't minded to tinker with and optimize my inner loops, I might have never discovered that the java.lang.Math routines are a lot faster than the equivalent Processing functions (functions like abs, max, etc). Back when I was messing with 80x86 assembly code, I did the same. It's all about having a somewhat accurate model of what goes on inside. With Processing it's the model that the functions are probably wrapped in something. With assembly code it's (among other things) the model that the order of independent instructions matters for purposes of pipelining and preventing AGI-stalls (I dunno if those are still relevant, this was the 386 era). When you get the hang of it, it's not really hard to experiment, measure, experiment, a few times and end up with spectacularly more performant code. And really, if the optimized result took a few days to calculate, and an hour of tinkering might save you another day or two, isn't that (often) worth it? Then there's the fact that I think it's fun :) Getting the code to work properly first, is a whole different process than spot-optimizing certain critical parts. I think it's refreshing to switch between these two different ways of looking at the same code.

panic 11y ago

I think there's an interesting distinction here between general-purpose software and single-use code. General-purpose software has many users with many kinds of data. Unless your domain is intrinsically limited, it's impossible to be "fast enough" -- you can always find some data that your code will take to long to process. There's always an incremental benefit to optimization, and with enough users, it's irresponsible to waste collective years of people's time by leaving performance on the table. What you've said makes perfect sense for one-off simulations, however. And in practice, optimizers are designed to make tight numerical code run fast. I don't think your use case will ever be ignored. It's just that most systems today are not made of tight loops of arithmetic operations. We need more tools to optimize the kind of code that actually makes our computers slow: code that allocates and deallocates memory, code that transforms data between different formats, code that constructs and operates on complex data structures -- that sort of thing.

valm- 11y ago
I don't entirely disagree, but I would like to make one point. Taking 1 millisecond from 1000 people, does not translate to 1 second of lost human productivity. There's no way to collect all those little slices of time and they are not appreciable individually. (Assuming those milliseconds are sufficiently spread out, i.e., not 1 ms out of every 2, or something like that)
- modeless 11y ago
  > Taking 1 millisecond from 1000 people, does not translate to 1 second of lost human productivity I believe that it does. If you average over a large population even tiny imperceptible increases in e.g. page load time have a measurable effect. Perhaps 1000 people isn't a large enough sample to measure the effect from 1ms delay considering all the confounding factors at play, but I absolutely believe you could see it with precise enough measurement and a large enough sample.
  - Rebelgecko 11y ago
    Reminds me of this semi-relevant story about improving the boot times of the original Macintosh — http://www.folklore.org/StoryView.py?story=Saving_Lives.txt http://www.folklore.org/StoryView.py?story=Saving_Lives.txt
    - gaius 11y ago
      The irony being that Jobs himself limited the memory in the original Mac, so it would have to go to disk more often!
      - jedrek 11y ago
        Source?
        gaius 11y ago
        Its a fairly well known story. The engineers ignored him and built in capability for 512k anyway. http://www.informationweek.com/mobile/mobile-devices/steve-jobs-jerry-pournelle-reflects/d/d-id/1099842 http://www.informationweek.com/mobile/mobile-devices/steve-j... He also hated fans and expansion slots.
  - darkmighty 11y ago
    Hmm I think the heart of the question is whether productivity is linear with time. I guess he's right that, for small values of t, productivity is highly superlinear. So giving 1 person an extra minute has an impact on that person, but because of superlinearity, if you distribute that among 60 persons the extra second won't allow anything new, like an original thought or whatever.
chriswarbo 11y ago
> There's always an incremental benefit to optimization, and with enough users, it's irresponsible to waste collective years of people's time by leaving performance on the table. I could just as easily claim that skilled developer's time is a precious resource, and it's irresponsible to waste it squeezing out performance when so much software still remains unwritten.
- bostik 11y ago
  There's another side to that, as well. Developers' time is expensive, and the time it takes to get feedback from code runs adds up when you have lots of developers. Especially if the roundtrip times are short enough not to allow switching between tasks, but long enough to cause constant "wall time waits". I'd be more than willing to have couple of extremely skilled developers profile and optimise the common code so that the feedback loop becomes shorter for everyone. That way more of the yet-unwritten software might just be tackled. My old boss once had me spend more than a week packaging and streamlining our access VPN configurations. When I asked him about it some time later, he explained that even though I was away from billable projects for that time, it would have cost a whole lot more overall to have all our developers and managers massage their setups every time something changed. Large figures are sums of lots of small figures...

gaius 11y ago

A typical compiler takes -O -O2 and so on flags for how much optimization you want. But what I would really like is a -O60s meaning, do as much optimization as you can in 60s and we're good to go. For the production version I might give it -O1h or -O12h and run it overnight. I am not aware of any product that does this.

TheLoneWolfling 11y ago
The one issue with this type of compilation is that it encourages hisenbugs. You add debugging code, it can't spend as much time optimizing, and all of a sudden it is producing the correct results again. But none the less, I agree.

tripzilch 11y ago

I agree it's a trade-off, it always is :) But if the goal is to write performant code, there's a few more variables to consider. An important one, IMO, is experience and practice. Optimization can be found in the weirdest places. My latest experience is not with C or assembly instruction crunching, but with Processing/Java. If I weren't minded to tinker with and optimize my inner loops, I might have never discovered that the java.lang.Math routines are a lot faster than the equivalent Processing functions (functions like abs, max, etc). Back when I was messing with 80x86 assembly code, I did the same. It's all about having a somewhat accurate model of what goes on inside. With Processing it's the model that the functions are probably wrapped in something. With assembly code it's (among other things) the model that the order of independent instructions matters for purposes of pipelining and preventing AGI-stalls (I dunno if those are still relevant, this was the 386 era). When you get the hang of it, it's not really hard to experiment, measure, experiment, a few times and end up with spectacularly more performant code. And really, if the optimized result took a few days to calculate, and an hour of tinkering might save you another day or two, isn't that (often) worth it? Then there's the fact that I think it's fun :) Getting the code to work properly first, is a whole different process than spot-optimizing certain critical parts. I think it's refreshing to switch between these two different ways of looking at the same code.