3 ms·

A professional gambler ought not ask this question. A professional gambler ought instead ask: how am I doing in the average? Hence the flaw in this research. It

A professional gambler ought not ask this question. A professional gambler ought instead ask: how am I doing in the average? Hence the flaw in this research. It’s bad gambling. I should think "not exactly." A professional gambler would take a look at each game, and determine if the expected payoff makes it worth playing. Here is where the analogy falls down. Not every analogue of "game" is the same! I worked for awhile for a company that was effectively a JIT VM vendor, and we found scenarios where our JIT could outperform things like a naively written YACC/LEX compiler. (Or a web templating engine. If we cranked up the size of the JIT code cache that ran at speeds expected of a C implementation.)

pizlonator 4y ago
No need to overthink the analogy. Pro gamblers choose their games while VM developers don’t. The analogy works for those games the gambler did happen to play. Let me give you more concrete data. On JSC we did tune for microbenchmarks in addition to macrobenchmarks. But the microbenchmarks would show pathologies not seen in the macrobenchmarks, those pathologies were unpredictable (you breathe on the VM and the behavior changes), and anyway real programs were more like macrobenchmarks than microbenchmarks. So - based on experience we switched to using the microbenchmarks only if they reliably reproduced behavior that we first observed in a macrobenchmarks. Got a macrobenchmark regression? See if any microbenchmarks also regressed in hopes of finding something easier to study. Otherwise the microbenchmarks were just whack-a-mole - if you focus on one of them you end up making changes that hurt real stuff. Sort of like if a gambler focused too much on one game, they’d end up losing in the average.
- stcredzero 4y ago
  Pro gamblers choose their games while VM developers don’t. Again, your answer is very informed, but very much in the VM developer POV. Pro gamblers can and should choose their games. Pro developers can and should also choose their games to make sure their expected payoff works out. While VM developers can "play" from the POV of the "house," and handle things in aggregate, we applicaiton developers choosing to use a VM or not are very much invested in our "next game."

pizlonator 4y ago

No need to overthink the analogy. Pro gamblers choose their games while VM developers don’t. The analogy works for those games the gambler did happen to play. Let me give you more concrete data. On JSC we did tune for microbenchmarks in addition to macrobenchmarks. But the microbenchmarks would show pathologies not seen in the macrobenchmarks, those pathologies were unpredictable (you breathe on the VM and the behavior changes), and anyway real programs were more like macrobenchmarks than microbenchmarks. So - based on experience we switched to using the microbenchmarks only if they reliably reproduced behavior that we first observed in a macrobenchmarks. Got a macrobenchmark regression? See if any microbenchmarks also regressed in hopes of finding something easier to study. Otherwise the microbenchmarks were just whack-a-mole - if you focus on one of them you end up making changes that hurt real stuff. Sort of like if a gambler focused too much on one game, they’d end up losing in the average.

stcredzero 4y ago
Pro gamblers choose their games while VM developers don’t. Again, your answer is very informed, but very much in the VM developer POV. Pro gamblers can and should choose their games. Pro developers can and should also choose their games to make sure their expected payoff works out. While VM developers can "play" from the POV of the "house," and handle things in aggregate, we applicaiton developers choosing to use a VM or not are very much invested in our "next game."