6 ms·

This article contains a logical error. The premise is JITs are hard to benchmark and keep good performance on. That's true. But the alternative is bad performa

This article contains a logical error. The premise is JITs are hard to benchmark and keep good performance on. That's true. But the alternative is bad performance all the time (the JITS fall back to interpretation, after all). What's the value in having clearly understood bad performance? If you care enough about performance that you need to understand it, surely you care about the absolute level of performance.

patrec 7y ago
False dichotomy. No one should be using shitty byte code interpreters like (c)python for real production work. Or at least stop whining about global warming. The real alternative is AOT. Also in many cases having it's much better to trade off average performance for lower variance.
- pjmlp 7y ago
  The real alternative is using toolchains that offer both, JIT/AOT.
  - patrec 7y ago
    Well, as long as we can agree that shitty bytecode interpreters are not the way to go ;) I think AOT/JIT as a dichotomy is stupid, for what it's worth. Why shouldn't I have e.g. both profile guided optimization and JIT in the same program?
    - pjmlp 7y ago
      That we surely agree, interpreters are only good as one is ramping up a language. or doesn't plan to use it beyond basic scripting or learning activities. You can have both on the same toolchain, and plenty language do have them. However in some scenarios, like constrained embedded devices, you are left with AOT only. By the way, modern Android is all three, interpreter, AOT and JIT with PGO.
- jashmatthews 7y ago
  The vast majority of the time it doesn't matter. Most of us don't write performance critical code. Even games use bytecode VMs for scripting. You'd be better off spending the money saved on planting trees. If you write a web app in Ruby using a similarly light weight frameworks as you would using Go, you'll be lucky to get 3x the throughput out of the Go app. There are also ways to more efficiently utilize hardware by using bytecode than distributing binaries. Using bytecode verification rather than an opaque binary, you can pack thousands of different web applications into a single process. This is an application of webassembly which IMHO is much more promising than in-browser use.
  - patrec 7y ago
    > The vast majority of the time it doesn't matter. I keep hearing this all the time and keep seeing all the time that it isn't true. Do you know of a single comparable Go web app that's not 10-100x less wasteful than goddamn gitlab? Most of the pathological sources of CO2 cloud emissions are a combination of terrible architecture and slow as molasses scripting languages like python or ruby. But these are not orthogonal. In theory gitlab could probably be 100x less wasteful even when written in Ruby. In practice many bad architectural decisions are far more likely to occur with Ruby and Python than a statically typed language (although Java may serve an interesting counter-example). > Using bytecode verification rather than an opaque binary, you can pack thousands of different web applications into a single process Can you expand on what problem this solves? Anyway, the problem isn't the bytecode, it's the shitty interpreter. There are plenty of bytecode based languages that are within < 10x of C.
    - jashmatthews 7y ago
      Yeah, absolutely. I've worked on some incredibly poorly performing Go services. The idea that a team struggling to deliver on a roadmap and keep up application performance while using a high level language like Ruby will magically be able to do it while using a lower level language is just a complete fallacy. It never happens! People get bogged down. They don't write low allocation code. Nobody has any time to turn on the compiler warnings for escape analysis. Your benchmarks end up a year out of date and don't even run anymore. Architecture becomes an afterthought. Function/method calls end up becoming even slower gRPC calls as the team struggles to box up complexity and extracts more services. WRT bytecode verification it's better just to read this: https://www.fastly.com/blog/announcing-lucet-fastly-native-webassembly-compiler-runtime https://www.fastly.com/blog/announcing-lucet-fastly-native-w... There's nothing shitty about the CRuby bytecode VM. It's all a question of resourcing. You're doing a huge disservice to all the people who worked very hard to make CRuby 2.7 5-15x faster than 1.8 and it detracts from a valid discussion about reducing the CO2 footprint of datacenters around the world.
lmm 7y ago
An unreliable optimisation can be worse than no optimisation, because if you understand your performance you can take steps to prepare appropriately (e.g. maybe you scale horizontally, or put some time into manual optimisation). If your code suddenly gets 10x slower in production for no clear reason, that's a lot worse than your code having been running 10x slower from day 1.
gridlockd 7y ago
On the server side, the alternative is an AOT compiled language.
- pjmlp 7y ago
  Many languages do offer both toolchains (AOT / JIT), including Java and .NET. The alternative is to leave JavaScript to the browser, where it belongs.
  - ninkendo 7y ago
    Could you provide some more detail/links about AOT-compiled Java?
    - imtringued 7y ago
      GraalVM native image but it has some major limitations.
    - pjmlp 7y ago
      As referred in another comments. Back to 2000's, ou had several commercial players, the most well known ones were Aicas, PTC, IBM, Aonix, J/Rockit, ... In what concerns Android, it went to a couple of ways, in Android 5 introduced AOT compilation, but it did not scale to do that on device, when multiple applications where installed, with Android 7 Google introduced another approach. Basically a mix of interpreted using a hand writter interpreter in Assembly, followed by a JIT, and then AOT compiling with PGO information gathered from the JIT, https://source.android.com/devices/tech/dalvik/jit-compiler https://source.android.com/devices/tech/dalvik/jit-compiler There are also a couple of Google IO related talks. Back to standard Java implementations, not only you have those commercial vendors, IBM and Oracle have finally made AOT a free feature so you get it on OpenJDK and OpenJ9. Besides AOT compiling, you also have the possibility to use a JIT cache between execution runs instead. This is sometimes a better approach, because you cannot use AOT on code that depends on reflection, unless you tell the compiler about all the classes that it doesn't see, but are also required to land on the executable. Most well known Java related conferences (JaX, Devoxx, NDC, Java ONE, CodeOne, JVM Language Summit) have a couple of AOT related talks every now and then.

patrec 7y ago

False dichotomy. No one should be using shitty byte code interpreters like (c)python for real production work. Or at least stop whining about global warming. The real alternative is AOT. Also in many cases having it's much better to trade off average performance for lower variance.

pjmlp 7y ago
The real alternative is using toolchains that offer both, JIT/AOT.
- patrec 7y ago
  Well, as long as we can agree that shitty bytecode interpreters are not the way to go ;) I think AOT/JIT as a dichotomy is stupid, for what it's worth. Why shouldn't I have e.g. both profile guided optimization and JIT in the same program?
  - pjmlp 7y ago
    That we surely agree, interpreters are only good as one is ramping up a language. or doesn't plan to use it beyond basic scripting or learning activities. You can have both on the same toolchain, and plenty language do have them. However in some scenarios, like constrained embedded devices, you are left with AOT only. By the way, modern Android is all three, interpreter, AOT and JIT with PGO.
jashmatthews 7y ago
The vast majority of the time it doesn't matter. Most of us don't write performance critical code. Even games use bytecode VMs for scripting. You'd be better off spending the money saved on planting trees. If you write a web app in Ruby using a similarly light weight frameworks as you would using Go, you'll be lucky to get 3x the throughput out of the Go app. There are also ways to more efficiently utilize hardware by using bytecode than distributing binaries. Using bytecode verification rather than an opaque binary, you can pack thousands of different web applications into a single process. This is an application of webassembly which IMHO is much more promising than in-browser use.
- patrec 7y ago
  > The vast majority of the time it doesn't matter. I keep hearing this all the time and keep seeing all the time that it isn't true. Do you know of a single comparable Go web app that's not 10-100x less wasteful than goddamn gitlab? Most of the pathological sources of CO2 cloud emissions are a combination of terrible architecture and slow as molasses scripting languages like python or ruby. But these are not orthogonal. In theory gitlab could probably be 100x less wasteful even when written in Ruby. In practice many bad architectural decisions are far more likely to occur with Ruby and Python than a statically typed language (although Java may serve an interesting counter-example). > Using bytecode verification rather than an opaque binary, you can pack thousands of different web applications into a single process Can you expand on what problem this solves? Anyway, the problem isn't the bytecode, it's the shitty interpreter. There are plenty of bytecode based languages that are within < 10x of C.
  - jashmatthews 7y ago
    Yeah, absolutely. I've worked on some incredibly poorly performing Go services. The idea that a team struggling to deliver on a roadmap and keep up application performance while using a high level language like Ruby will magically be able to do it while using a lower level language is just a complete fallacy. It never happens! People get bogged down. They don't write low allocation code. Nobody has any time to turn on the compiler warnings for escape analysis. Your benchmarks end up a year out of date and don't even run anymore. Architecture becomes an afterthought. Function/method calls end up becoming even slower gRPC calls as the team struggles to box up complexity and extracts more services. WRT bytecode verification it's better just to read this: https://www.fastly.com/blog/announcing-lucet-fastly-native-webassembly-compiler-runtime https://www.fastly.com/blog/announcing-lucet-fastly-native-w... There's nothing shitty about the CRuby bytecode VM. It's all a question of resourcing. You're doing a huge disservice to all the people who worked very hard to make CRuby 2.7 5-15x faster than 1.8 and it detracts from a valid discussion about reducing the CO2 footprint of datacenters around the world.

lmm 7y ago

An unreliable optimisation can be worse than no optimisation, because if you understand your performance you can take steps to prepare appropriately (e.g. maybe you scale horizontally, or put some time into manual optimisation). If your code suddenly gets 10x slower in production for no clear reason, that's a lot worse than your code having been running 10x slower from day 1.

gridlockd 7y ago

On the server side, the alternative is an AOT compiled language.

pjmlp 7y ago
Many languages do offer both toolchains (AOT / JIT), including Java and .NET. The alternative is to leave JavaScript to the browser, where it belongs.
- ninkendo 7y ago
  Could you provide some more detail/links about AOT-compiled Java?
  - imtringued 7y ago
    GraalVM native image but it has some major limitations.
  - pjmlp 7y ago
    As referred in another comments. Back to 2000's, ou had several commercial players, the most well known ones were Aicas, PTC, IBM, Aonix, J/Rockit, ... In what concerns Android, it went to a couple of ways, in Android 5 introduced AOT compilation, but it did not scale to do that on device, when multiple applications where installed, with Android 7 Google introduced another approach. Basically a mix of interpreted using a hand writter interpreter in Assembly, followed by a JIT, and then AOT compiling with PGO information gathered from the JIT, https://source.android.com/devices/tech/dalvik/jit-compiler https://source.android.com/devices/tech/dalvik/jit-compiler There are also a couple of Google IO related talks. Back to standard Java implementations, not only you have those commercial vendors, IBM and Oracle have finally made AOT a free feature so you get it on OpenJDK and OpenJ9. Besides AOT compiling, you also have the possibility to use a JIT cache between execution runs instead. This is sometimes a better approach, because you cannot use AOT on code that depends on reflection, unless you tell the compiler about all the classes that it doesn't see, but are also required to land on the executable. Most well known Java related conferences (JaX, Devoxx, NDC, Java ONE, CodeOne, JVM Language Summit) have a couple of AOT related talks every now and then.