22 ms·

Or just use something like ThinLTO. You don't need a JIT just to get cross-module inlining, which is all you're really talking about here. There's no actual "sp

Or just use something like ThinLTO. You don't need a JIT just to get cross-module inlining, which is all you're really talking about here. There's no actual "specialization" in play, just basic inlining + dead branch elimination. Trivial to do this statically at build time instead. So much so this is even commonly done in JIT'd languages like Java (Proguard & R8 say hello) Most JITs also have the downside of they need to optimize for compilation performance, and as such typically do not do as thorough a job optimizing as something like offline LLVM. Or you end up with fourth-tier JITs like webkit's FTL. And now instead of just needing to wait for the JIT to think your function is interesting you now need to wait for a second and finally third JIT pass. JITs really primarily just shine in the rapid iteration cycle. Conceivably what you really want is a language where debug builds are JIT and release builds are full AOT with LTO. Not entirely unlike Dart's behavior in Flutter. Or even how Java behaves on Android to an extent. There doesn't seem to be any particularly significant reason why this pairing isn't more broadly applicable and used. There's also no real reason you can't have zero-cost abstractions AND a JIT if you want. Hell, in theory this is what you get with C++ in a webasm environment. This are not exactly opposing technologies.

pron 7y ago
> get cross-module inlining, which is all you're really talking about here. Not at all. > There's no actual "specialization" in play, just basic inlining + dead branch elimination. Devirtualization + inlining + dead branch elimination is an instance of specialization, and an extremely effective one at that, but other forms of specializations apply. > Trivial to do this statically at build time instead. No. Some things are undecidable and/or would lead to code explosion. E.g., it's often hard to statically prove that a subroutine that takes an int would always receive the number 2, and specializing it for all ints would cause code explosion (again, contrived example but with real-world counterparts). > There's also no real reason you can't have zero-cost abstractions AND a JIT if you want. Hell, in theory this is what you get with C++ in a webasm environment. This are not exactly opposing technologies. Of course. But I was talking about two very different philosophies that greatly affect language design.
- kllrnohj 7y ago
  > Not at all. For your example? Yes, it was. > Devirtualization + inlining + dead branch elimination is an instance of specialization Your example did not involve any devirtualization. > No. Some things are undecidable and/or would lead to code explosion. E.g., it's often hard to statically prove that a subroutine that takes an int would always receive the number 2, and specializing it for all ints would cause code explosion (again, contrived example but with real-world counterparts). Again, your example did not have any such issues. And if it was hard to statically prove a subroutine that takes an int always receives 2 then guess what? A JIT wouldn't do it, either. They don't actually specialize that much for runtime information because gathering that information would destroy performance. Devirtualization? Sure, limited in scope, and huge payoffs to be had for doing it. All arguments analyzed & binned? Good god no. That'd be ludicrous in resource usage required, and the payoffs would be incredibly hard to quantify in any meaningful way. Take your example - all it would do is eliminate a single branch. If it was so common to take a single avenue that it could be eliminated, then pretty much every branch predictor would do it anyway. So... why specialize it?
  - pron 7y ago
    > Take your example - all it would do is eliminate a single branch. If it was so common to take a single avenue that it could be eliminated, then pretty much every branch predictor would do it anyway. So... why specialize it? I don't think you've actually read the comment. The particular example is just a demonstration of a principle. Obviously that particular case is too simple to be interesting. > A JIT wouldn't do it, either. Again, you've completely missed the point (read the original comment, perhaps?). In any event, a JIT could and would do it in some cases; e.g.: https://gist.github.com/chrisseaton/4464807d93b972813e49 https://gist.github.com/chrisseaton/4464807d93b972813e49
    - kllrnohj 7y ago
      > In any event, a JIT could and would do it in some cases; e.g.: https://gist.github.com/chrisseaton/4464807d93b972813e49 https://gist.github.com/chrisseaton/4464807d93b972813e49 Uh, what? That's just a basic constant fold optimization that can be statically handled? It even has examples showing anything that would require runtime information (like mutable strings) don't get optimized.
the8472 7y ago
Hotspot performs inlining + DCE/CSE/costant folding/... in a loop, they call it incremental inlining. I.e. inline, eliminate, now you have new inlining opportunities because the code dropped below your max inlining size, rinse & repeat. I think rustc does not iterate llvm optimization passes and thus even with lto leaves opportunities on the table.
- kllrnohj 7y ago
  > I think rustc does not iterate llvm optimization passes and thus even with lto leaves opportunities on the table. I think you'd need to support that with some evidence. If there's any code that hotspot manages to out-optimize LLVM on that'd be a fascinating investigation. In practice I think you'll be shocked how well GCC & LLVM make code vanish vs. Hotspot. It's one of the major reasons benchmarking C/C++ code is so damn hard, the optimizer tends to eliminate the entire thing.
  - pron 7y ago
    > In practice I think you'll be shocked how well GCC & LLVM make code vanish vs. Hotspot. First of all, HotSpot is a VM, not a compiler. It has several compilers, among them C2 (the default optimizing compiler), Graal, but also LLVM (Zing is a HotSpot fork VM which uses an LLVM-based JIT called Falcon: https://www.azul.com/products/zing/falcon-jit-compiler/ https://www.azul.com/products/zing/falcon-jit-compiler/) + several GCs + other stuff. Second, I think you may be shocked at the comparison of LLVM to C2: https://youtu.be/O87PaWkXlZ0 https://youtu.be/O87PaWkXlZ0 (they come up about the same, each winning/losing on different things). And that is when the optimization can be proven; whatever optimization you have, a JIT can apply them more aggressively.
  - the8472 7y ago
    Note that it is up to the compiler frontend to choose which LLVM optimization passes to call, in which order and how often. But neither clang[0] nor rustc[1] do something that resembles hotspot C2's incremental inlining[2]. It is fairly expensive, so hotspot's benefit here is that it only has to spend the optimization cycles on the hottest code. > It's one of the major reasons benchmarking C/C++ code is so damn hard, the optimizer tends to eliminate the entire thing. This is totally a thing in java too. That's why the java microbenchmark harness (JMH) provides helpers that can inhibit DCE, constant propagation and inlining [0] https://www.youtube.com/watch?v=s4wnuiCwTGU https://www.youtube.com/watch?v=s4wnuiCwTGU [1] https://github.com/rust-lang/rust/issues/44041 https://github.com/rust-lang/rust/issues/44041 [2] https://bugs.openjdk.java.net/browse/JDK-8005071 https://bugs.openjdk.java.net/browse/JDK-8005071
wolf550e 7y ago
ThinLTO does not work across shared libraries loaded at runtime. HotSpot does.

pron 7y ago

> get cross-module inlining, which is all you're really talking about here. Not at all. > There's no actual "specialization" in play, just basic inlining + dead branch elimination. Devirtualization + inlining + dead branch elimination is an instance of specialization, and an extremely effective one at that, but other forms of specializations apply. > Trivial to do this statically at build time instead. No. Some things are undecidable and/or would lead to code explosion. E.g., it's often hard to statically prove that a subroutine that takes an int would always receive the number 2, and specializing it for all ints would cause code explosion (again, contrived example but with real-world counterparts). > There's also no real reason you can't have zero-cost abstractions AND a JIT if you want. Hell, in theory this is what you get with C++ in a webasm environment. This are not exactly opposing technologies. Of course. But I was talking about two very different philosophies that greatly affect language design.

kllrnohj 7y ago
> Not at all. For your example? Yes, it was. > Devirtualization + inlining + dead branch elimination is an instance of specialization Your example did not involve any devirtualization. > No. Some things are undecidable and/or would lead to code explosion. E.g., it's often hard to statically prove that a subroutine that takes an int would always receive the number 2, and specializing it for all ints would cause code explosion (again, contrived example but with real-world counterparts). Again, your example did not have any such issues. And if it was hard to statically prove a subroutine that takes an int always receives 2 then guess what? A JIT wouldn't do it, either. They don't actually specialize that much for runtime information because gathering that information would destroy performance. Devirtualization? Sure, limited in scope, and huge payoffs to be had for doing it. All arguments analyzed & binned? Good god no. That'd be ludicrous in resource usage required, and the payoffs would be incredibly hard to quantify in any meaningful way. Take your example - all it would do is eliminate a single branch. If it was so common to take a single avenue that it could be eliminated, then pretty much every branch predictor would do it anyway. So... why specialize it?
- pron 7y ago
  > Take your example - all it would do is eliminate a single branch. If it was so common to take a single avenue that it could be eliminated, then pretty much every branch predictor would do it anyway. So... why specialize it? I don't think you've actually read the comment. The particular example is just a demonstration of a principle. Obviously that particular case is too simple to be interesting. > A JIT wouldn't do it, either. Again, you've completely missed the point (read the original comment, perhaps?). In any event, a JIT could and would do it in some cases; e.g.: https://gist.github.com/chrisseaton/4464807d93b972813e49 https://gist.github.com/chrisseaton/4464807d93b972813e49
  - kllrnohj 7y ago
    > In any event, a JIT could and would do it in some cases; e.g.: https://gist.github.com/chrisseaton/4464807d93b972813e49 https://gist.github.com/chrisseaton/4464807d93b972813e49 Uh, what? That's just a basic constant fold optimization that can be statically handled? It even has examples showing anything that would require runtime information (like mutable strings) don't get optimized.

the8472 7y ago

Hotspot performs inlining + DCE/CSE/costant folding/... in a loop, they call it incremental inlining. I.e. inline, eliminate, now you have new inlining opportunities because the code dropped below your max inlining size, rinse & repeat. I think rustc does not iterate llvm optimization passes and thus even with lto leaves opportunities on the table.

kllrnohj 7y ago
> I think rustc does not iterate llvm optimization passes and thus even with lto leaves opportunities on the table. I think you'd need to support that with some evidence. If there's any code that hotspot manages to out-optimize LLVM on that'd be a fascinating investigation. In practice I think you'll be shocked how well GCC & LLVM make code vanish vs. Hotspot. It's one of the major reasons benchmarking C/C++ code is so damn hard, the optimizer tends to eliminate the entire thing.
- pron 7y ago
  > In practice I think you'll be shocked how well GCC & LLVM make code vanish vs. Hotspot. First of all, HotSpot is a VM, not a compiler. It has several compilers, among them C2 (the default optimizing compiler), Graal, but also LLVM (Zing is a HotSpot fork VM which uses an LLVM-based JIT called Falcon: https://www.azul.com/products/zing/falcon-jit-compiler/ https://www.azul.com/products/zing/falcon-jit-compiler/) + several GCs + other stuff. Second, I think you may be shocked at the comparison of LLVM to C2: https://youtu.be/O87PaWkXlZ0 https://youtu.be/O87PaWkXlZ0 (they come up about the same, each winning/losing on different things). And that is when the optimization can be proven; whatever optimization you have, a JIT can apply them more aggressively.
- the8472 7y ago
  Note that it is up to the compiler frontend to choose which LLVM optimization passes to call, in which order and how often. But neither clang[0] nor rustc[1] do something that resembles hotspot C2's incremental inlining[2]. It is fairly expensive, so hotspot's benefit here is that it only has to spend the optimization cycles on the hottest code. > It's one of the major reasons benchmarking C/C++ code is so damn hard, the optimizer tends to eliminate the entire thing. This is totally a thing in java too. That's why the java microbenchmark harness (JMH) provides helpers that can inhibit DCE, constant propagation and inlining [0] https://www.youtube.com/watch?v=s4wnuiCwTGU https://www.youtube.com/watch?v=s4wnuiCwTGU [1] https://github.com/rust-lang/rust/issues/44041 https://github.com/rust-lang/rust/issues/44041 [2] https://bugs.openjdk.java.net/browse/JDK-8005071 https://bugs.openjdk.java.net/browse/JDK-8005071

wolf550e 7y ago

ThinLTO does not work across shared libraries loaded at runtime. HotSpot does.