4 ms·

From the title, I expected to find one of the frequently called case statements was unoptimized and they fixed it. Instead, the fix was rewriting all case state

From the title, I expected to find one of the frequently called case statements was unoptimized and they fixed it. Instead, the fix was rewriting all case statements in a way the javascript engine would optimize. Very frustrating if you're unaware of the obscure rules around it.

benreesman 4y ago
This kind of stuff is usually at least somewhat frustrating in the moment, but it’s just the nature of things: we’re always mapping some task onto some machine and it wouldn’t be such a great story if it wasn’t such an elegant hack. This one is particularly interesting because it’s one VM mapping onto a second VM and the impedance mismatch is very meta. My only nitpick with a great read is that I want to know a little more gory details of how the automation was done, it could easily have involved a third or even fourth VM!
yarg 4y ago
Not quite - the javascript engine failed to optimise reasonable situations, so the dude translated to a code structure (arrays of functions) that didn't require optimisation. Essentially he was forced to manually code what should've been done under the hood anyway.
- shadowofneptune 4y ago
  I'm unsure. They're different enough in structure that it feels to me that it should be a conscious decision. In a switch statememt, there's usually one large scope, while in the function table there's a lot of little scopes. The pathway through the code during execution is different ad well. Massive switch statements are also not reasonable situations from the perspective of a compiler, as they see little use outside of VMs.
  - andreareina 4y ago
    VMs aren't exactly rare though, and are a case (heh) of when you really want all the optimization the compiler can give.
    - naniwaduni 4y ago
      In the grand scheme of things, VMs and interpreters are a class of program that tend toward perhaps surprisingly pathological behaviour, among programs people are likely to write.
    - yarg 4y ago
      O(1 for small n; n otherwise) just seems like a poor decision.
  - yarg 4y ago
    > They're different enough in structure... Sure, I mean more in terms of producing a compiled form with O(1) rather than O(n) performance. > Massive switch statements are also not reasonable situations from the perspective of a compiler, as they see little use outside of VMs. I'm not really following this logic, massive switch statements are those that can benefit most significantly from the generation of jump tables. And what needs to be done to generate them doesn't differ significantly based upon the branch count (when you hit case 0x100 you'll need to switch to 16-bit, so binary size will be impacted). The fact that it's primary use case is reasonably niche is fairly irrelevant given there's bugger all compiler overhead required for support. Now from a code perspective, I would say that a statement that large probably isn't ideal (depending on the size of your case blocks).
    - shadowofneptune 4y ago
      > Now from a code perspective, I would say that a statement that large probably isn't ideal (depending on the size of your case blocks). This is what I had meant, yes. Optimizing such large blocks is quite difficult. A case label in C and C-like languages does not even need a block, making it harder. > The fact that it's primary use case is reasonably niche is fairly irrelevant given there's bugger all compiler overhead required for support. Even static C compilers still struggle with optimizing the switch-in-for-loop pattern, let alone a JIT compiler. Clang does the best job nowadays, but it still lags behind other methods like computed goto or continuation-passing. The Python VM will switch to computed goto if it's available on the C compiler. Best primer on the subject I have found, even if old: https://www.complang.tuwien.ac.at/forth/threaded-code.html https://www.complang.tuwien.ac.at/forth/threaded-code.html My own benchmarks from last year (note that these toy VMs may not be applicable to larger ones): https://github.com/shadowofneptune/threaded-code-benchmark https://github.com/shadowofneptune/threaded-code-benchmark It does take work to make a faster VM, it feels unreasonable to expect a compiler to do a good job at optimizing it without effort on the writer's side.
- xena 4y ago
  she* the author is female.
  - yarg 4y ago
    If the sex of the author had any bearing on my interpretation of the article, I might actually give a damn. But it doesn't; so I don't.
eyelidlessness 4y ago
But knowing how the JIT deoptimizes is a good thing for anyone programming perf-sensitive code for that JIT. Even developing a sense of where branchy logic might deopt is a good thing, even if your code might not be perf-sensitive but could be called in a perf-sensitive call stack, because it’ll give you a good heuristic for when optimization is premature. Optimizing for the JIT isn’t and shouldn’t be a guiding principle for every dev, but it’s harmless or good for us to be aware.
- saagarjha 4y ago
  > Optimizing for the JIT isn’t and shouldn’t be a guiding principle for every dev, but it’s harmless or good for us to be aware. No, it can be quite harmful: you could end up overoptimizing for one engine and ruin performance on another engine, or even a different version of the same engine. When making non-trivial optimizations that are designed around the quirks of the optimizing compiler you should be very careful about the tradeoffs and also what the penalty will be if the optimization breaks in the future.
  - pjmlp 4y ago
    Just like it happens in C when driven by assembly output on a specific compiler version.
    - saagarjha 4y ago
      Yep, exactly. There are very few codebases that can actually write code directly to the optimizer and they are usually developed very closely with a specific compiler. For other cases usually you can trust that a number of basic optimizations can be performed and then be cautious about what the various compilers can do. As you gain more experience you can slowly expand the set of things you would expect to be "easy" for the compiler–when those passes don't run, they're generally considered to be bugs.
deleted 4y ago
[deleted]
pjmlp 4y ago
It is no different from compiled languages, that is one of the reasons why most compilers regardless of the language, allow to look into the generated Assembly.
- IshKebab 4y ago
  It's not fundamentally different but it is generally a lot easier to predict what will be fast with e.g. Clang than V8.
  - anonymoushn 4y ago
    For inner loops of interpreters in low-level languages, unless you are using some labels-are-values extension, it's often tough to convince the thing to generate the right code.
  - pjmlp 4y ago
    Only if one has a PhD in ISO C semantics and compiler optimization handling of UB. What is fast in clang might be totally crap on xlC, or even plain clang vNext, for example.
- mysterydip 4y ago
  The difference is control. You can choose which compiler to use for your project, even an outdated version if it works for your use case. But you have no control over what JIT engine visits your site.
  - pjmlp 4y ago
    Partially true, most of the time many dev teams don't control what is on the CI/CD servers, or what compilers are being used when shipping libraries.

benreesman 4y ago

This kind of stuff is usually at least somewhat frustrating in the moment, but it’s just the nature of things: we’re always mapping some task onto some machine and it wouldn’t be such a great story if it wasn’t such an elegant hack. This one is particularly interesting because it’s one VM mapping onto a second VM and the impedance mismatch is very meta. My only nitpick with a great read is that I want to know a little more gory details of how the automation was done, it could easily have involved a third or even fourth VM!

yarg 4y ago

Not quite - the javascript engine failed to optimise reasonable situations, so the dude translated to a code structure (arrays of functions) that didn't require optimisation. Essentially he was forced to manually code what should've been done under the hood anyway.

shadowofneptune 4y ago
I'm unsure. They're different enough in structure that it feels to me that it should be a conscious decision. In a switch statememt, there's usually one large scope, while in the function table there's a lot of little scopes. The pathway through the code during execution is different ad well. Massive switch statements are also not reasonable situations from the perspective of a compiler, as they see little use outside of VMs.
- andreareina 4y ago
  VMs aren't exactly rare though, and are a case (heh) of when you really want all the optimization the compiler can give.
  - naniwaduni 4y ago
    In the grand scheme of things, VMs and interpreters are a class of program that tend toward perhaps surprisingly pathological behaviour, among programs people are likely to write.
  - yarg 4y ago
    O(1 for small n; n otherwise) just seems like a poor decision.
- yarg 4y ago
  > They're different enough in structure... Sure, I mean more in terms of producing a compiled form with O(1) rather than O(n) performance. > Massive switch statements are also not reasonable situations from the perspective of a compiler, as they see little use outside of VMs. I'm not really following this logic, massive switch statements are those that can benefit most significantly from the generation of jump tables. And what needs to be done to generate them doesn't differ significantly based upon the branch count (when you hit case 0x100 you'll need to switch to 16-bit, so binary size will be impacted). The fact that it's primary use case is reasonably niche is fairly irrelevant given there's bugger all compiler overhead required for support. Now from a code perspective, I would say that a statement that large probably isn't ideal (depending on the size of your case blocks).
  - shadowofneptune 4y ago
    > Now from a code perspective, I would say that a statement that large probably isn't ideal (depending on the size of your case blocks). This is what I had meant, yes. Optimizing such large blocks is quite difficult. A case label in C and C-like languages does not even need a block, making it harder. > The fact that it's primary use case is reasonably niche is fairly irrelevant given there's bugger all compiler overhead required for support. Even static C compilers still struggle with optimizing the switch-in-for-loop pattern, let alone a JIT compiler. Clang does the best job nowadays, but it still lags behind other methods like computed goto or continuation-passing. The Python VM will switch to computed goto if it's available on the C compiler. Best primer on the subject I have found, even if old: https://www.complang.tuwien.ac.at/forth/threaded-code.html https://www.complang.tuwien.ac.at/forth/threaded-code.html My own benchmarks from last year (note that these toy VMs may not be applicable to larger ones): https://github.com/shadowofneptune/threaded-code-benchmark https://github.com/shadowofneptune/threaded-code-benchmark It does take work to make a faster VM, it feels unreasonable to expect a compiler to do a good job at optimizing it without effort on the writer's side.
xena 4y ago
she* the author is female.
- yarg 4y ago
  If the sex of the author had any bearing on my interpretation of the article, I might actually give a damn. But it doesn't; so I don't.

eyelidlessness 4y ago

But knowing how the JIT deoptimizes is a good thing for anyone programming perf-sensitive code for that JIT. Even developing a sense of where branchy logic might deopt is a good thing, even if your code might not be perf-sensitive but could be called in a perf-sensitive call stack, because it’ll give you a good heuristic for when optimization is premature. Optimizing for the JIT isn’t and shouldn’t be a guiding principle for every dev, but it’s harmless or good for us to be aware.

saagarjha 4y ago
> Optimizing for the JIT isn’t and shouldn’t be a guiding principle for every dev, but it’s harmless or good for us to be aware. No, it can be quite harmful: you could end up overoptimizing for one engine and ruin performance on another engine, or even a different version of the same engine. When making non-trivial optimizations that are designed around the quirks of the optimizing compiler you should be very careful about the tradeoffs and also what the penalty will be if the optimization breaks in the future.
- pjmlp 4y ago
  Just like it happens in C when driven by assembly output on a specific compiler version.
  - saagarjha 4y ago
    Yep, exactly. There are very few codebases that can actually write code directly to the optimizer and they are usually developed very closely with a specific compiler. For other cases usually you can trust that a number of basic optimizations can be performed and then be cautious about what the various compilers can do. As you gain more experience you can slowly expand the set of things you would expect to be "easy" for the compiler–when those passes don't run, they're generally considered to be bugs.

deleted 4y ago

[deleted]

pjmlp 4y ago

It is no different from compiled languages, that is one of the reasons why most compilers regardless of the language, allow to look into the generated Assembly.

IshKebab 4y ago
It's not fundamentally different but it is generally a lot easier to predict what will be fast with e.g. Clang than V8.
- anonymoushn 4y ago
  For inner loops of interpreters in low-level languages, unless you are using some labels-are-values extension, it's often tough to convince the thing to generate the right code.
- pjmlp 4y ago
  Only if one has a PhD in ISO C semantics and compiler optimization handling of UB. What is fast in clang might be totally crap on xlC, or even plain clang vNext, for example.
mysterydip 4y ago
The difference is control. You can choose which compiler to use for your project, even an outdated version if it works for your use case. But you have no control over what JIT engine visits your site.
- pjmlp 4y ago
  Partially true, most of the time many dev teams don't control what is on the CI/CD servers, or what compilers are being used when shipping libraries.