4 ms·

> Questions: "Many optimisations that can really help high level managed languages like Kotlin simply don’t apply to C++": Which optimisations? I understood th

> Questions: "Many optimisations that can really help high level managed languages like Kotlin simply don’t apply to C++": Which optimisations? I understood the JVM does an dynamic analysis of the code when running, so it can compile to better performance. For example it knows which branch of an if it is more likely to be executed, so it can compile that code to native in a way the CPU jump prediction executes the branch that will probably executed at the end. (it is hard to explain without more knowledge but CPUs do optimizations in branches and usually executed one branch before even know the result of the condition, so you sometimes save some cycles, with the dynamic analysis you know which branch is more often executed). I guess the author is talking about this type of things

mhh__ 9y ago
That's called profile-guided optimisation and most good optimising compilers do it. Not a particularly good example, however the JVM can do it in a slightly different way because doing PGO on every build/test or in the wild is prohibitively annoying (e.g. The JVM can do it for each time it runs, rather than when the compiler is told to)
- jorgemf 9y ago
  You cannot do it before compiling the final executable, you have to do it while running. You can simulate the profiling but you will get the best results when you profile while executing. And I guess it is easier to do profiling in high level language than in low level. Another example is to unroll loops. Probably you know more than my about optimization, but I think the idea is clear here. It is easier to get better gains with high level code and profiling in real time, than with low level code and simulating the profiling.
- mike_hearn 9y ago
  PGO in C++ is in practice only rarely used, due to the difficulty of setting it up and incorporating it into the build cycle. As a consequence compiler writers rarely focus on it and C++ is designed on the assumption that optimisation is always done statically. So there's a kind of feedback loop there. Java is designed on the assumption that JITC is available and JVM designers focus more on PGO as a result.
  - DannyBee 9y ago
    "PGO in C++ is in practice only rarely used, due to the difficulty of setting it up and incorporating it into the build cycle." This is pretty much false these days. Most folks who care about it use it. You were right 10 years ago. At this point, there are enough companies where 10% perf is tens or hundreds of millions of dollars, and so the world changed. "As a consequence compiler writers rarely focus on it and C++ is designed on the assumption that optimisation is always done statically." Also false as a consequence of the above :) "Java is designed on the assumption that JITC is available and JVM designers focus more on PGO as a result. " But they have less time to perform any of the expensive opts pgo enables. so ...
    - mike_hearn 9y ago
      I'm sure Google does these days, though when I was there PGO wasn't widely used. But even Chrome only started using PGO late last year (15% speed boost!), and Chrome is one of the most intensively optimised apps out there. So I think it's fair to say that most C++ apps built by mere mortal companies probably still don't use it. But I'd be keen to see data showing I'm wrong. Perhaps MSVC++ or LLVM leaves some trace of PGO behind in the binary, and a Windows/Mac machine with lots of apps installed could be scanned to get an idea (for desktop usage at least).
      - chubot 9y ago
        I just saw that Python's Makefile has support for PGO builds: https://github.com/python/cpython/blob/master/Makefile.pre.in#L479 https://github.com/python/cpython/blob/master/Makefile.pre.i... Not sure when it was added, but Python 2.7 has it. It's not clear to me how to tell if say my Ubuntu binary is built with PGO, but it's something I'm looking into.
      - DannyBee 9y ago
        "I'm sure Google does these days, though when I was there PGO wasn't widely used". Yes it was. Maybe you didn't know, but it was. "and Chrome is one of the most intensively optimised apps out there. " This isn't really true actually. So if that's your counter-argument, ... "So I think it's fair to say that most C++ apps built by mere mortal companies probably still don't use it" No, it isn't :) Really. So far you've cited one example of someone being late to the party. " But I'd be keen to see data showing I'm wrong." What form do you want it to take?
- zamalek 9y ago
  AOT PGO is fantastic tech! However, it can only optimize for the profile that you set up for it. Deploy it on a million machines with a million use cases and some of them aren't going to match up with your profile. In these cases it may very well degrade performance. So you might try to profile scenarios that hit all codepoints equally, by doing so you would defeat the purpose of PGO (and likely nullify it). JIT PGO is adaptive to the workload that the process is experiencing. It is likely impossible to achieve this with AOT compilation.
  - DannyBee 9y ago
    "JIT PGO is adaptive to the workload that the process is experiencing. It is likely impossible to achieve this with AOT compilation. " Of course it's not :) You just generate different binaries for each use case. Truthfully, the "million machines/million use cases" story is pretty non-realistic these days (except in the above case!) Most people run mostly-specialized services for mostly-specialized use-cases. This is because it's necessary for performance in practice anyway.
soulbadguy 9y ago
> I understood the JVM does an dynamic analysis of the code when running, so it can compile to better performance. The argument that JIT having more information and therefore should/could be able to produce better code never really materialized in most the of mainline JITs (JVM,.net etc...). From my point of view, there is a lot of reasons for this : - Most VM starts with higher level languages than native systems. And most of the "runtime informations" is not used to produce faster "native code" as used to efficiently deconstructs some of the higher abstractions (think devirtualization, unboxing, moving stuff from the heap to the stack etc...) - Because Jitting happens at runtime, most jit shy away from complex compiler transformations : So you have more runtime information but less time to actually do something useful with that information. - Tracking, recording, updating and all the booking involve in collecting any runtime information also limit "quality" of the profile in practice further more reducing their impact. > I guess the author is talking about this type of things I don't think so, the author is probably talking about optimization which lowers higher level constructs, like unboxing etc...
- _old_dude_ 9y ago
  You have forgotten several important analysis routinely done by JITs that significantly speedup Java: - deep inlining, by default the inlining depth of a JVM is around 10, you can not have this depth of inlining with a statically compiled language. - prune non taken branches (and exception blocks), so you do not fill the instruction cache with code never executed. - branch re-ordering, the code executed more often can be read from top to bottom, less executed code is offloaded to a side block. - CPU instructions selection at runtime. - lock profiling, generate different codes if the lock is contented or not. BTW .Net do not do any code profiling, so can not use most of these optimizations.
- mike_hearn 9y ago
  The argument that JIT having more information and therefore should/could be able to produce better code never really materialized in most the of mainline JITs Disabling JIT PGO in HotSpot loses you 15-20% performance, so I think it did materialise. Because Jitting happens at runtime, most jit shy away from complex compiler transformations I wonder what you classify as complex? Escape analysis driven scalar replacement, PICs, superword optimisations, lock elimination, automatic hardware transactional memory usage all seem very complex to me. HotSpot uses two JITs to trade off speed vs complexity, C1 does a very fast compile with only fast optimisations, C2 will revisit very hot methods and do the extensive optimisation of them.
DannyBee 9y ago
"I understood the JVM does an dynamic analysis of the code when running, so it can compile to better performance. " Considering this was one of the main points of llvm in the first place, and it's had a jit since day 1 .. http://llvm.org/pubs/2004-01-30-CGO-LLVM.html http://llvm.org/pubs/2004-01-30-CGO-LLVM.html "This paper describes LLVM (Low Level Virtual Machine), a compiler framework designed to support transparent, lifelong program analysis and transformation for arbitrary programs, by providing high-level information to compiler transformations at compile-time, link-time, run-time, and in idle time between runs."

mhh__ 9y ago

That's called profile-guided optimisation and most good optimising compilers do it. Not a particularly good example, however the JVM can do it in a slightly different way because doing PGO on every build/test or in the wild is prohibitively annoying (e.g. The JVM can do it for each time it runs, rather than when the compiler is told to)

jorgemf 9y ago
You cannot do it before compiling the final executable, you have to do it while running. You can simulate the profiling but you will get the best results when you profile while executing. And I guess it is easier to do profiling in high level language than in low level. Another example is to unroll loops. Probably you know more than my about optimization, but I think the idea is clear here. It is easier to get better gains with high level code and profiling in real time, than with low level code and simulating the profiling.
mike_hearn 9y ago
PGO in C++ is in practice only rarely used, due to the difficulty of setting it up and incorporating it into the build cycle. As a consequence compiler writers rarely focus on it and C++ is designed on the assumption that optimisation is always done statically. So there's a kind of feedback loop there. Java is designed on the assumption that JITC is available and JVM designers focus more on PGO as a result.
- DannyBee 9y ago
  "PGO in C++ is in practice only rarely used, due to the difficulty of setting it up and incorporating it into the build cycle." This is pretty much false these days. Most folks who care about it use it. You were right 10 years ago. At this point, there are enough companies where 10% perf is tens or hundreds of millions of dollars, and so the world changed. "As a consequence compiler writers rarely focus on it and C++ is designed on the assumption that optimisation is always done statically." Also false as a consequence of the above :) "Java is designed on the assumption that JITC is available and JVM designers focus more on PGO as a result. " But they have less time to perform any of the expensive opts pgo enables. so ...
  - mike_hearn 9y ago
    I'm sure Google does these days, though when I was there PGO wasn't widely used. But even Chrome only started using PGO late last year (15% speed boost!), and Chrome is one of the most intensively optimised apps out there. So I think it's fair to say that most C++ apps built by mere mortal companies probably still don't use it. But I'd be keen to see data showing I'm wrong. Perhaps MSVC++ or LLVM leaves some trace of PGO behind in the binary, and a Windows/Mac machine with lots of apps installed could be scanned to get an idea (for desktop usage at least).
    - chubot 9y ago
      I just saw that Python's Makefile has support for PGO builds: https://github.com/python/cpython/blob/master/Makefile.pre.in#L479 https://github.com/python/cpython/blob/master/Makefile.pre.i... Not sure when it was added, but Python 2.7 has it. It's not clear to me how to tell if say my Ubuntu binary is built with PGO, but it's something I'm looking into.
    - DannyBee 9y ago
      "I'm sure Google does these days, though when I was there PGO wasn't widely used". Yes it was. Maybe you didn't know, but it was. "and Chrome is one of the most intensively optimised apps out there. " This isn't really true actually. So if that's your counter-argument, ... "So I think it's fair to say that most C++ apps built by mere mortal companies probably still don't use it" No, it isn't :) Really. So far you've cited one example of someone being late to the party. " But I'd be keen to see data showing I'm wrong." What form do you want it to take?
zamalek 9y ago
AOT PGO is fantastic tech! However, it can only optimize for the profile that you set up for it. Deploy it on a million machines with a million use cases and some of them aren't going to match up with your profile. In these cases it may very well degrade performance. So you might try to profile scenarios that hit all codepoints equally, by doing so you would defeat the purpose of PGO (and likely nullify it). JIT PGO is adaptive to the workload that the process is experiencing. It is likely impossible to achieve this with AOT compilation.
- DannyBee 9y ago
  "JIT PGO is adaptive to the workload that the process is experiencing. It is likely impossible to achieve this with AOT compilation. " Of course it's not :) You just generate different binaries for each use case. Truthfully, the "million machines/million use cases" story is pretty non-realistic these days (except in the above case!) Most people run mostly-specialized services for mostly-specialized use-cases. This is because it's necessary for performance in practice anyway.

soulbadguy 9y ago

> I understood the JVM does an dynamic analysis of the code when running, so it can compile to better performance. The argument that JIT having more information and therefore should/could be able to produce better code never really materialized in most the of mainline JITs (JVM,.net etc...). From my point of view, there is a lot of reasons for this : - Most VM starts with higher level languages than native systems. And most of the "runtime informations" is not used to produce faster "native code" as used to efficiently deconstructs some of the higher abstractions (think devirtualization, unboxing, moving stuff from the heap to the stack etc...) - Because Jitting happens at runtime, most jit shy away from complex compiler transformations : So you have more runtime information but less time to actually do something useful with that information. - Tracking, recording, updating and all the booking involve in collecting any runtime information also limit "quality" of the profile in practice further more reducing their impact. > I guess the author is talking about this type of things I don't think so, the author is probably talking about optimization which lowers higher level constructs, like unboxing etc...

_old_dude_ 9y ago
You have forgotten several important analysis routinely done by JITs that significantly speedup Java: - deep inlining, by default the inlining depth of a JVM is around 10, you can not have this depth of inlining with a statically compiled language. - prune non taken branches (and exception blocks), so you do not fill the instruction cache with code never executed. - branch re-ordering, the code executed more often can be read from top to bottom, less executed code is offloaded to a side block. - CPU instructions selection at runtime. - lock profiling, generate different codes if the lock is contented or not. BTW .Net do not do any code profiling, so can not use most of these optimizations.
mike_hearn 9y ago
The argument that JIT having more information and therefore should/could be able to produce better code never really materialized in most the of mainline JITs Disabling JIT PGO in HotSpot loses you 15-20% performance, so I think it did materialise. Because Jitting happens at runtime, most jit shy away from complex compiler transformations I wonder what you classify as complex? Escape analysis driven scalar replacement, PICs, superword optimisations, lock elimination, automatic hardware transactional memory usage all seem very complex to me. HotSpot uses two JITs to trade off speed vs complexity, C1 does a very fast compile with only fast optimisations, C2 will revisit very hot methods and do the extensive optimisation of them.

DannyBee 9y ago

"I understood the JVM does an dynamic analysis of the code when running, so it can compile to better performance. " Considering this was one of the main points of llvm in the first place, and it's had a jit since day 1 .. http://llvm.org/pubs/2004-01-30-CGO-LLVM.html http://llvm.org/pubs/2004-01-30-CGO-LLVM.html "This paper describes LLVM (Low Level Virtual Machine), a compiler framework designed to support transparent, lifelong program analysis and transformation for arbitrary programs, by providing high-level information to compiler transformations at compile-time, link-time, run-time, and in idle time between runs."