3 ms·

For what it's worth, the paper does acknowledge this view in the section 1.3 (and your quote of Knuth does appear in a Bernstein's talk, referenced from that se

For what it's worth, the paper does acknowledge this view in the section 1.3 (and your quote of Knuth does appear in a Bernstein's talk, referenced from that section). The main reason the authors still pursue this direction is, to my best understanding, that: * As you pointed out, we are moving to higher-level constructs and languages (this is directly addressed by a Pugh's talk as well); and * "[...] in any case getting high-performance executables out of high-level languages seems to fundamentally require aggressive compiler optimization". In the other words, the widespread use of higher-level languages does not prove that the (automatic) optimization is not needed, but rather that the needs for higher-level languages are very high despite of the cost. And indeed, all languages you've mentioned have highly optimized interpreters or JIT compilers, albeit crafted by humans. I also feel that the paper importantly discusses the speed and correctness of code optimization along with the resulting optimized code (Thesis 1), and it is not just for the fully-automatic optimizer as you detest. The correctness of such program transformations will eventually require many tools mentioned in the paper.

mpweiher 8y ago
> section 1.3 I missed the reference to Bernstein (and Pugh), thanks! Though they seem to think he was kidding. I don't think so. A little tongue-in-cheek, maybe, but not kidding, because what Bernstein and Pugh discuss are very real. Most code is cold, most optimizations here are a waste of compiler resources, better to compile to a simple/compact bytecode to keep code-densities high and memory clean. (See also Google's recent experience with re-implementing their JS engines). Some code is hot, here most compilers are not good enough, though we will take what they can give us, not necessarily because you need assembler (as Bernstein says), but because higher level optimizations that do 100x - 1000x are needed when coming from high-level code. Often, the "cold" code is setup code, whereas the hot code is in loops that get run once set up, see the Scripted Components pattern. Of course, once you follow that pattern, you run into the Two Languages problem that Julia is trying to solve. > getting high-performance executables out of high-level languages seems to fundamentally require aggressive compiler optimization". If you (a) treat the compiler (or runtime) as a black box and (b) don't permit multiple levels (as per Knuth) than I agree, sort of. Except that you don't actually reliably get high performance that way. You may sometimes get high peak performance, if you're lucky. Now they seem to argue that their approach would do away with the variability of that approach, by consistently/reliably burning through layers of abstraction. "finding optimizations using aggressive search algorithms and SMT solvers that are, unlike humans, not easy to fool with superficial changes of representation." I wish them cough sufficiently smart compiler cough luck, particularly when the goal is also to make/keep compile times reasonable. I also have a hard time with the phrase "unlike humans, not easy to fool", because what's easy to fool here is the compiler, and the "superficial changes of representation", as the changes needed are often not superficial. To me, an alternative solution to the the Scripted Components / Two Languages problem with progressive refinement and the ability to specify constraints seems like the right path. So instead of, for example, writing high level (OO?) code and hoping that the compiler/runtime will do the escape analysis and put things in registers instead of allocating them on the heap, you write the same code and then (a) specify that you don't want heap allocations in a particular region (b) have the compiler tell you whether that was possible and (c) assist you in putting in annotations/transformations that accomplish this goal and remain visible in the code, possibly separate from the other other code.

mpweiher 8y ago

> section 1.3 I missed the reference to Bernstein (and Pugh), thanks! Though they seem to think he was kidding. I don't think so. A little tongue-in-cheek, maybe, but not kidding, because what Bernstein and Pugh discuss are very real. Most code is cold, most optimizations here are a waste of compiler resources, better to compile to a simple/compact bytecode to keep code-densities high and memory clean. (See also Google's recent experience with re-implementing their JS engines). Some code is hot, here most compilers are not good enough, though we will take what they can give us, not necessarily because you need assembler (as Bernstein says), but because higher level optimizations that do 100x - 1000x are needed when coming from high-level code. Often, the "cold" code is setup code, whereas the hot code is in loops that get run once set up, see the Scripted Components pattern. Of course, once you follow that pattern, you run into the Two Languages problem that Julia is trying to solve. > getting high-performance executables out of high-level languages seems to fundamentally require aggressive compiler optimization". If you (a) treat the compiler (or runtime) as a black box and (b) don't permit multiple levels (as per Knuth) than I agree, sort of. Except that you don't actually reliably get high performance that way. You may sometimes get high peak performance, if you're lucky. Now they seem to argue that their approach would do away with the variability of that approach, by consistently/reliably burning through layers of abstraction. "finding optimizations using aggressive search algorithms and SMT solvers that are, unlike humans, not easy to fool with superficial changes of representation." I wish them cough sufficiently smart compiler cough luck, particularly when the goal is also to make/keep compile times reasonable. I also have a hard time with the phrase "unlike humans, not easy to fool", because what's easy to fool here is the compiler, and the "superficial changes of representation", as the changes needed are often not superficial. To me, an alternative solution to the the Scripted Components / Two Languages problem with progressive refinement and the ability to specify constraints seems like the right path. So instead of, for example, writing high level (OO?) code and hoping that the compiler/runtime will do the escape analysis and put things in registers instead of allocating them on the heap, you write the same code and then (a) specify that you don't want heap allocations in a particular region (b) have the compiler tell you whether that was possible and (c) assist you in putting in annotations/transformations that accomplish this goal and remain visible in the code, possibly separate from the other other code.