6 ms·
Eh, with the benefit of 14 years of hindsight, I want to push back on some of the things in that talk. (Context: I work on SpiderMonkey.) First, all the stuff
by IainIreland 5y ago
Eh, with the benefit of 14 years of hindsight, I want to push back on some of the things in that talk. (Context: I work on SpiderMonkey.)
First, all the stuff about tracing and trace trees is kind of obsolete. SpiderMonkey abandoned TraceMonkey a long time ago. (To the best of my knowledge, V8 never implemented a tracing JIT at all.) The problem with tracing is that you can get really good performance when everything goes right, but it's brittle. There's a reference in the talk to how implementations of the Game of Life can have exponential blow-up, for example. You can usually fix any individual pathological case, but the inherently exponential number of possible paths through a program make it difficult to completely eliminate weird performance cliffs.
If your goal is to maximize performance on a known set of benchmarks, go wild. If you want a robust engine that can handle whatever weird code the web is throwing at you, tracing JITs are (as far as I can tell) a dead end.
(Counterpoint: LuaJIT seems to be doing alright with tracing, although it may just solve the problem by punting to the programmer: https://github.com/lukego/blog/issues/29 https://github.com/lukego/blog/issues/29. That's more feasible when you don't have multiple engines with performance cliffs in subtly different places.)
Second, the idea that JIT-compiled code can be faster than AOT-compiled code has been floating around for a long time, but I don't think it really holds in the general case. Doing work at runtime isn't free: not just time spent compiling, but also time spent profiling and validating that your speculative optimizations continue to be correct.
SpiderMonkey had a top-tier optimizing compiler, IonMonkey, that got pretty darn close to native code on hot benchmark loops. We tracked whole-program information to ensure that type checks could be elided in inner loops. (For example, if the `x` property of a certain set of objects only ever contained 32-bit integers, then we could unbox it without checking the type. If any code elsewhere ever stored a non-integer value in that property, we would notice and invalidate the optimized code.)
We threw IonMonkey away, because it was too brittle. In practice, real-world code falls off the happy path often enough that we got better performance by accepting that even your highly optimized JS code will include some runtime checks. Invalidation and recompilation are real costs. So is the upkeep of all the global data necessary to support Ion. There's an engineering tradeoff between pushing up the performance ceiling and bringing up the performance floor; we've been happy with our choice to shift focus more towards the latter. Our numbers are down on artificial benchmarks, but it seems to have paid off in real-world performance. (Also, bugs in the optimizing compiler are significantly less likely to be exploitable.)
A lot of really smart people have done some incredible work on the JVM. Nevertheless, I'm still not aware of any code written in Java instead of (say) C++ or Rust because Java was faster. I think it's more accurate to say that JIT compilation can be fast enough that the other advantages of the language can make it the right choice.
- asiachick 5y agoDo you have some good examples of how this "real world" perf if tested? I often write micro benchmarks to see if something is faster one way or another. I often find one browser or another to be 3x to 10x faster in a certain micro benchmark. A big issue is, most websites don't need perf AFAICT. Of the sites I use regularly (HN, Stack Overflow, GMail, Gdocs, GSheets, Facebook, Messenger, Slack, Reddit, github, maybe only gsheets and gdocs need any real perf. The place where perf is needed is things like three.js, playcanvas, babylon.js, unity -> html, etc... And on those AFAIK, Chrome almost always wins, probably, because it's GPU support is multi-process
- IainIreland 5y agoThere's no magic answer here. We track various metrics (page load time, responsiveness, gc pause time, and so on) in telemetry. (You can poke around at telemetry.mozilla.org.) We have some page load benchmarks that load a recorded copy of various pages and track how long it takes to load. Sometimes capturing a profile of a particular site can help. Of the big JS benchmarks, we've found Speedometer to be the least bad, because there was at least an effort to mimic actual websites. Microbenchmarks are tricky because it's very easy to measure something other than what you intend: differences in inlining heuristics, say. In the long run, I expect performance-critical websites to migrate to webassembly.
- meheleventyone 5y agoI have to second this, we have a JS based game engine and Firefox is very much noticeably slower than Safari and Chrome in terms of JS performance. I agree with Iain below that in the long run this will mean moving more code to wasm so hopefully the tooling there improves. Right now all it does is move users of performance critical web-first software away from Firefox. Although I’d note that performance is utterly important for normal applications as well. Being noticeably snappy and prompt is great for UX.
- hajile 5y agoA couple theoretical questions. Has there been any consideration to create AI to work around pathological cases in tracing? If compiling is done on a secondary thread, aren't you then reaping all the benefits of JIT optimization while still not paying the penalty on the actual JS execution thread? Why throw away optimized code? Why not add a type check on the parameters and then dispatch to the highly-optimized code version based on the parameter count and types?