9 ms·

I'd be interested to see this talk, because as far as i know, it's pretty far off base. 1. He says" As computation has become cheaper, users have corresponding

I'd be interested to see this talk, because as far as i know, it's pretty far off base. 1. He says" As computation has become cheaper, users have correspondingly expanded the volume of data that they are handling, and optimization remains a critical challenge for the occasional "hot spots" in the code."" Except, uh, a lot of people have applications whose profiles are mostly flat, because they've spent a lot of time optimizing them. We build optimizing compilers that can speed up these apps anyway. 2."Have compilers become so smart that they automatically turn clean high-level code for these hot spots into optimized code, removing the need for humans to be optimization experts? The reality, unfortunately, is very much the opposite: general-purpose "optimizing" compilers are falling farther and farther behind the actual capabilities of modern processors." This is just flat out wrong. That said, Daniel is a very smart (and obviously pretty opinionated :P) guy, so i'm really interested to see what he has to say.

readerrr 12y ago
On 2.: Maybe he is talking about branch prediction vs lack of this feature in compilers. What do I mean by that is that optimizing compilers( at least in c ) don't optimize based on measurements from actual programs. A compiler that would run the program for a certain amount of time, to profile, could find the weak spots and optimize.
- mansr 12y ago
  GCC can do that.
  - readerrr 12y ago
    You are mistaken, unless you can prove me wrong. Care to elaborate?
    - dalke 12y ago
      I assume mansr refers to profile guided optimizations in gcc and many other compilers. See http://en.wikipedia.org/wiki/Profile-guided_optimization http://en.wikipedia.org/wiki/Profile-guided_optimization.
    - thoughtpolice 12y ago
      GCC has supported `-fprofile-generate` and `-fprofile-use` for profile-guided optimization since (at least[1]) GCC 3.4, which was released over 10 years ago: https://gcc.gnu.org/onlinedocs/gcc-3.4.0/gcc/Optimize-Options.html https://gcc.gnu.org/onlinedocs/gcc-3.4.0/gcc/Optimize-Option... Firefox is an example of an application that ships production binaries with PGO. [1] It was actually supported earlier than this, but I didn't search further; the 3.4.0 manual says "New -fprofile-generate and -fprofile-use command-line options to simplify the use of profile feedback", implying it was there before in a less convenient form.
    - fche 12y ago
      Or rather, "I may be mistaken in my grand assertion, but I'll place the burden of proof on you".
- ericfontaine 12y ago
  In addition to GCC, a bunch of other compilers including Intel and Visual Studio do profile based optimization, as well as many dynamic compilers, such as firfox javascript or Java VMs. Profil-guided opitmization has been studied for decades: https://en.wikipedia.org/wiki/Profile-guided_optimization https://en.wikipedia.org/wiki/Profile-guided_optimization
ericfontaine 12y ago
I would add for 2. that there are newer architectures, especially low power and various experimental parallel ones, that could use better optimizing compilers.
haberman 12y ago
I don't know Dan's work/thinking well enough to anticipate what he might talk about. But if someone wanted me to speak on this prompt, I would talk about modern examples where optimizing compilers significantly underperform hand-written assembly: - LuaJIT's interpreter (here is Mike Pall's famous post giving a detailed explanation of why this is, incidentally prompted by a question I asked: http://lua-users.org/lists/lua-l/2011-02/msg00742.html http://lua-users.org/lists/lua-l/2011-02/msg00742.html) - x264, which has extensive acceleration implemented in assembly for a/v codecs (they even built their own assembly abstraction layer: http://x264dev.multimedia.cx/archives/191 http://x264dev.multimedia.cx/archives/191) - GMP, which has extensive acceleration implemented in assembly for arbitrary-precision arithmetic. I wouldn't go so far as to argue that any of this implies the death of optimizing compilers, so I will be very interested to see what Dan has to say.
chubot 12y ago
Regarding #2, it's conventional wisdom that you can't beat a compiler, but I'm sure DJB is going to show examples where he beats the compiler significantly. That doesn't surprise me for two reasons: 1) DJB is a great programmer. 2) Compilers are so complex that there are lots of non-obvious reasons that they miss optimizations. I'm not an expert, but it appears to me that, in the last 10 years, processors have gotten complex at higher rate than compilers have gotten better. There's just more room to do nontrivial things that have come up only recently. I found this interesting portable ASM DSL on DJB's site: http://cr.yp.to/qhasm.html http://cr.yp.to/qhasm.html Honest question: Can you write C that compiles to what his qasm programs do with a modern C compiler? (I suspect the answer is no, because DJB writes tons of C and wouldn't have written it othewise) Older perf notes on his site: http://cr.yp.to/hardware/x86.html http://cr.yp.to/hardware/x86.html On the other hand, DJB writes a very particular type of software (crypto and mathematical code lately), so I suspect his experience won't be that generalizable. But it will be correct and important, because his niches are important.
- Dewie 12y ago
  > Regarding #2, it's conventional wisdom that you can't beat a compiler, but I'm sure DJB is going to show examples where he beats the compiler significantly. Is the fact that a compiler can beat you sometimes[1] really that controversial? And he better have a large class of problems in which he can beat the compiler, without using too much time[2]. Or else the title is just needlessly provocative. And thirdly: many of just rely on the compiler doing a good enough job. Not a better job than we would do if we were more careful. [1] Or often times. Depends on who you ask, and in what domains. [2] Or else you can just say that a programmer can beat a compiler given enough time, where time can be arbitrarily long. But all of us have some kind of time constraint.
mpweiher 12y ago
Hmm...the abstract matches my experience pretty closely. A lot of the non-critical code is perfectly fine, performance-wise, in bytecode interpreted Squeak Smalltalk or even Ruby. On the other hand, perf critical code tends to require re-thinking your abstractions and often your layering, which can give you several orders of magnitude of performance improvement (I've frequently had 10x - 1000x). So I don't need the optimizer for the first part, and it doesn't really help me with the second part. It can and sometimes does take out some of the drudgery, but it can also get in the way by being somewhat unpredictable. What I find interesting is the approach talked about by VPRI of allowing manual optimization specifications sitting next to high-level code (some of the earliest AOP examples were like that). Less magic, more control, much more predictability and power.
vardump 12y ago
2 is very much true. Compilers are pretty clever using general purpose registers, but falling behind when it comes to using SIMD vectorization in non-vector context. A human "compiler" can beat best compilers by 2-10x pretty often by just using new CPU features compilers have yet to "learn".
nkurz 12y ago
> The reality, unfortunately, is very much the opposite: > general-purpose "optimizing" compilers are falling farther > and farther behind the actual capabilities of modern > processors. This is just flat out wrong. Saying this is wrong implies that the gap between ideal processor-specific assembly and generated code is closing, and that compilers today can achieve a higher percentage of potential performance than they could in the past. Do you have evidence that suggests this is true? Are you possibly conflating "optimizing the code as written including all unintentially specified corner cases" with "optimizing the production of the desired answer"? In my personal experience (which is mostly compression and scientific computing), I find that I can get significantly higher x64 performance from Haswell than from earlier Intel generations --- frequently more than twice as fast as I can do for Sandy Bridge, due to a combination of better memory interactions, fewer scheduling gotcha's, and wider integer vectors. Very rarely do I find that compiler generated code (Clang, GCC, icc) achieves this level of speedup, and even more rarely do I find that the same code gets optimal performance on both of these generations. Even more rarely does the generated assembly surprise me by exceeding my expectations of maximal performance. I'm not sure where Daniel is going with his presentation, but what I often find myself wanting is a non-brain-dead non-optimizing compiler. It makes me sad to know how to make code run fast, but be unable to keep the compiler from slowing it down. Obviously, this is a rare case, since only a tiny percentage of code will ever be allowed to be this highly optimized. But I sure wish that there was a way to return to "glorified assembly", and that I had a way to tell the compiler to keep its grubby little fingers off certain small sections of my consciously crafted code.
- DannyBee 12y ago
  "aying this is wrong implies that the gap between ideal processor-specific assembly and generated code is closing, and that compilers today can achieve a higher percentage of potential performance than they could in the past. Do you have evidence that suggests this is true? " Yes. We now do things like http://drona.csa.iisc.ernet.in/~uday/publications/pluto+.pdf http://drona.csa.iisc.ernet.in/~uday/publications/pluto+.pdf Which will fully, automatically, parallelize and optimize loop nests to the target architecture, including tiling, interchange, cache blocking, blah blah blah Basically anything you can think of. You can't do this by hand. You can do something by hand, maybe tiling, maybe unrolling, maybe interchange but the math is too complex for you to get it right in the general case and to combine all these things at once. We couldn't do this 20 years ago. We couldn't even approach the performance of most architectures. We can also vectorize whatever you want. The only hard part is calculating the cost vs benefit. Past that, now we actually have the opposite problem. We come so close to optimal on most architectures that we can't do much more without using NP complete algorithms instead of heuristics. We can only try to get little niggles here and there where the heuristics get slightly wrong answers. We decided we care more about compile speed than doing stuff like this. "Very rarely do I find that compiler generated code (Clang, GCC, icc) achieves this level of speedup, and even more rarely do I find that the same code gets optimal performance on both of these generations." I don't know your code, so i really can't speak to this at all, but it's the opposite experience for us. In fact, moving platforms now makes little performance difference. Getting the compiler up to snuff for that architecture does. If what you say is true, it just tells me you need to spend a small amount of time figuring out why the compiler is doing the wrong thing. It's almost certainly just a small bug or tweak somewhere.

readerrr 12y ago

On 2.: Maybe he is talking about branch prediction vs lack of this feature in compilers. What do I mean by that is that optimizing compilers( at least in c ) don't optimize based on measurements from actual programs. A compiler that would run the program for a certain amount of time, to profile, could find the weak spots and optimize.

mansr 12y ago
GCC can do that.
- readerrr 12y ago
  You are mistaken, unless you can prove me wrong. Care to elaborate?
  - dalke 12y ago
    I assume mansr refers to profile guided optimizations in gcc and many other compilers. See http://en.wikipedia.org/wiki/Profile-guided_optimization http://en.wikipedia.org/wiki/Profile-guided_optimization.
  - thoughtpolice 12y ago
    GCC has supported `-fprofile-generate` and `-fprofile-use` for profile-guided optimization since (at least[1]) GCC 3.4, which was released over 10 years ago: https://gcc.gnu.org/onlinedocs/gcc-3.4.0/gcc/Optimize-Options.html https://gcc.gnu.org/onlinedocs/gcc-3.4.0/gcc/Optimize-Option... Firefox is an example of an application that ships production binaries with PGO. [1] It was actually supported earlier than this, but I didn't search further; the 3.4.0 manual says "New -fprofile-generate and -fprofile-use command-line options to simplify the use of profile feedback", implying it was there before in a less convenient form.
  - fche 12y ago
    Or rather, "I may be mistaken in my grand assertion, but I'll place the burden of proof on you".
ericfontaine 12y ago
In addition to GCC, a bunch of other compilers including Intel and Visual Studio do profile based optimization, as well as many dynamic compilers, such as firfox javascript or Java VMs. Profil-guided opitmization has been studied for decades: https://en.wikipedia.org/wiki/Profile-guided_optimization https://en.wikipedia.org/wiki/Profile-guided_optimization

ericfontaine 12y ago

I would add for 2. that there are newer architectures, especially low power and various experimental parallel ones, that could use better optimizing compilers.

haberman 12y ago

I don't know Dan's work/thinking well enough to anticipate what he might talk about. But if someone wanted me to speak on this prompt, I would talk about modern examples where optimizing compilers significantly underperform hand-written assembly: - LuaJIT's interpreter (here is Mike Pall's famous post giving a detailed explanation of why this is, incidentally prompted by a question I asked: http://lua-users.org/lists/lua-l/2011-02/msg00742.html http://lua-users.org/lists/lua-l/2011-02/msg00742.html) - x264, which has extensive acceleration implemented in assembly for a/v codecs (they even built their own assembly abstraction layer: http://x264dev.multimedia.cx/archives/191 http://x264dev.multimedia.cx/archives/191) - GMP, which has extensive acceleration implemented in assembly for arbitrary-precision arithmetic. I wouldn't go so far as to argue that any of this implies the death of optimizing compilers, so I will be very interested to see what Dan has to say.

chubot 12y ago

Regarding #2, it's conventional wisdom that you can't beat a compiler, but I'm sure DJB is going to show examples where he beats the compiler significantly. That doesn't surprise me for two reasons: 1) DJB is a great programmer. 2) Compilers are so complex that there are lots of non-obvious reasons that they miss optimizations. I'm not an expert, but it appears to me that, in the last 10 years, processors have gotten complex at higher rate than compilers have gotten better. There's just more room to do nontrivial things that have come up only recently. I found this interesting portable ASM DSL on DJB's site: http://cr.yp.to/qhasm.html http://cr.yp.to/qhasm.html Honest question: Can you write C that compiles to what his qasm programs do with a modern C compiler? (I suspect the answer is no, because DJB writes tons of C and wouldn't have written it othewise) Older perf notes on his site: http://cr.yp.to/hardware/x86.html http://cr.yp.to/hardware/x86.html On the other hand, DJB writes a very particular type of software (crypto and mathematical code lately), so I suspect his experience won't be that generalizable. But it will be correct and important, because his niches are important.

Dewie 12y ago
> Regarding #2, it's conventional wisdom that you can't beat a compiler, but I'm sure DJB is going to show examples where he beats the compiler significantly. Is the fact that a compiler can beat you sometimes[1] really that controversial? And he better have a large class of problems in which he can beat the compiler, without using too much time[2]. Or else the title is just needlessly provocative. And thirdly: many of just rely on the compiler doing a good enough job. Not a better job than we would do if we were more careful. [1] Or often times. Depends on who you ask, and in what domains. [2] Or else you can just say that a programmer can beat a compiler given enough time, where time can be arbitrarily long. But all of us have some kind of time constraint.

mpweiher 12y ago

Hmm...the abstract matches my experience pretty closely. A lot of the non-critical code is perfectly fine, performance-wise, in bytecode interpreted Squeak Smalltalk or even Ruby. On the other hand, perf critical code tends to require re-thinking your abstractions and often your layering, which can give you several orders of magnitude of performance improvement (I've frequently had 10x - 1000x). So I don't need the optimizer for the first part, and it doesn't really help me with the second part. It can and sometimes does take out some of the drudgery, but it can also get in the way by being somewhat unpredictable. What I find interesting is the approach talked about by VPRI of allowing manual optimization specifications sitting next to high-level code (some of the earliest AOP examples were like that). Less magic, more control, much more predictability and power.

vardump 12y ago

2 is very much true. Compilers are pretty clever using general purpose registers, but falling behind when it comes to using SIMD vectorization in non-vector context. A human "compiler" can beat best compilers by 2-10x pretty often by just using new CPU features compilers have yet to "learn".

nkurz 12y ago

> The reality, unfortunately, is very much the opposite: > general-purpose "optimizing" compilers are falling farther > and farther behind the actual capabilities of modern > processors. This is just flat out wrong. Saying this is wrong implies that the gap between ideal processor-specific assembly and generated code is closing, and that compilers today can achieve a higher percentage of potential performance than they could in the past. Do you have evidence that suggests this is true? Are you possibly conflating "optimizing the code as written including all unintentially specified corner cases" with "optimizing the production of the desired answer"? In my personal experience (which is mostly compression and scientific computing), I find that I can get significantly higher x64 performance from Haswell than from earlier Intel generations --- frequently more than twice as fast as I can do for Sandy Bridge, due to a combination of better memory interactions, fewer scheduling gotcha's, and wider integer vectors. Very rarely do I find that compiler generated code (Clang, GCC, icc) achieves this level of speedup, and even more rarely do I find that the same code gets optimal performance on both of these generations. Even more rarely does the generated assembly surprise me by exceeding my expectations of maximal performance. I'm not sure where Daniel is going with his presentation, but what I often find myself wanting is a non-brain-dead non-optimizing compiler. It makes me sad to know how to make code run fast, but be unable to keep the compiler from slowing it down. Obviously, this is a rare case, since only a tiny percentage of code will ever be allowed to be this highly optimized. But I sure wish that there was a way to return to "glorified assembly", and that I had a way to tell the compiler to keep its grubby little fingers off certain small sections of my consciously crafted code.

DannyBee 12y ago
"aying this is wrong implies that the gap between ideal processor-specific assembly and generated code is closing, and that compilers today can achieve a higher percentage of potential performance than they could in the past. Do you have evidence that suggests this is true? " Yes. We now do things like http://drona.csa.iisc.ernet.in/~uday/publications/pluto+.pdf http://drona.csa.iisc.ernet.in/~uday/publications/pluto+.pdf Which will fully, automatically, parallelize and optimize loop nests to the target architecture, including tiling, interchange, cache blocking, blah blah blah Basically anything you can think of. You can't do this by hand. You can do something by hand, maybe tiling, maybe unrolling, maybe interchange but the math is too complex for you to get it right in the general case and to combine all these things at once. We couldn't do this 20 years ago. We couldn't even approach the performance of most architectures. We can also vectorize whatever you want. The only hard part is calculating the cost vs benefit. Past that, now we actually have the opposite problem. We come so close to optimal on most architectures that we can't do much more without using NP complete algorithms instead of heuristics. We can only try to get little niggles here and there where the heuristics get slightly wrong answers. We decided we care more about compile speed than doing stuff like this. "Very rarely do I find that compiler generated code (Clang, GCC, icc) achieves this level of speedup, and even more rarely do I find that the same code gets optimal performance on both of these generations." I don't know your code, so i really can't speak to this at all, but it's the opposite experience for us. In fact, moving platforms now makes little performance difference. Getting the compiler up to snuff for that architecture does. If what you say is true, it just tells me you need to spend a small amount of time figuring out why the compiler is doing the wrong thing. It's almost certainly just a small bug or tweak somewhere.