5 ms·

To have a meaningful "victory", you want to have one of those: 1. A generic JS JIT that runs generic JS faster than another generic JS JIT. 2. ASM.JS JIT

To have a meaningful "victory", you want to have one of those: 1. A generic JS JIT that runs generic JS faster than another generic JS JIT. 2. ASM.JS JIT that runs ASM.JS faster than another ASM.JS JIT. What we have here is something else: 3. A generic JS JIT that runs ASM.JS faster than another generic JS JIT. If people are not familiar with ASM.JS, the somewhat misleading (to the uninitiated) headline sounds more interesting than it is actually. It's not a significant milestone, as SpiderMonkey's ASM.JS implementation outperforms this one, and V8 isn't implementing ASM.JS in any way. As in at all. So of course it'll be slower. The purpose of ASM.JS is to allow a separate ASM.JS runtime to run ASM.JS code. The reason it happens to be a JS subset, is so you can have a fallback for backwards compatibility. Not to say this LLVM engine doesn't show potential. But implementing ASM.JS in LLVM is not the hard part of implementing a generic JS JIT. ASM.JS is entirely static, so it's comparatively trivial to JIT, it's literally an ASM to ASM transpiler.

flohofwoe 12y ago
IIRC, V8 had implemented optimizations that specifically target asm.js code patterns in the past (the performance gap for asm.js code between FF and Chrome definitely got a lot smaller in 2013), they just don't rely on the "use asm" keyword or do AOT compilation like Odinmonkey.
- lucian1900 12y ago
  It was more fixing bugs in their JIT that were only really exercised by asm.js-like code.
sanxiyn 12y ago
Edit: the particular quote I objected to was edited out by parent. I object to "all the dynamicness of the rest of the JS is what V8 excels at". V8 excels at a subset of dynamic JavaScript (namely those benefit from Crankshaft) and is not very good outside of the subset. Typescript benchmark included in Octane benchmark suite is a good example. In Google's own words: "This test measures the time TypeScript takes to compile itself and is a proxy of how well a VM handles complex and sizable Javascript applications". In particular, Crankshaft is not effective for this benchmark. (Check yourself by running it with Crankshaft disabled: the score actually improves because you save compilation effort.) And as you can check yourself or check on AreWeFastYet, JavaScriptCore beats V8 by 2x on this benchmark. I admit Crankshaft subset of JavaScript is large and interesting (certainly compared to asm.js subset) but it is not true that V8 excels at dynamic JavaScript, for sufficiently large value of dynamic. JavaScriptCore does.
- mantrax5 12y ago
  Noted. Sorry for removing this, I indeed said that, but I thought it's redundant so I removed it.
- mraleph 12y ago
  TypeScript benchmark is one of the Octane benchmarks that does not do warm up, so Crankshaft does not have chance to fully shine [which is totally fair given how TypeScript compiler is used: you don't warm it up, you just spawn it again and again]. It also has an extremely flat profile. So you can't simply say that "Crankshaft is not effective for it". Though potentially you can say "benefits of Crankshafting do not compensate for the cost of Crankshafting". If you enable warmup for it then score jumps by a factor of 2 for V8. JSC might be warming faster or simply have a better cost / benefit ratio at lower tiers. I doubt FTL kicks at all for this benchmark given how short running it is. V8 also deopts quite a bit on it, so it might be some pathological case... This is definitely something worth exploring and fixing on V8 side.
- zurn 12y ago
  > V8 excels at a subset of dynamic JavaScript (namely those benefit from Crankshaft) and is not very good outside of the subset As I understand it Crankshaft's claim to fame is the runtime type specialization, which does well if the code is hot and ends up using same types in all cases. Is this what you are referring to?
scriptproof 12y ago
Your comment contradicts itself: "Asm.js is a JS subset" and "V8 isn't implementing Asm.js in any way.". It is fully implemented, of course, it is only the "use asm.js" directive that is not. That prevents some control to signal non asm.js code in a JS file, and only that.
azakai 12y ago
I don't think there is a "victory" here. Just an important milestone for Safari/JavaScriptCore. They have done some amazing engineering here! In the broader context, this shows that the JSC approach of taking LLVM and using it as a JIT - called FTL - can beat Chrome/v8 on large long-running computational code. That means that LLVM is significantly better than v8's backend JIT at some things. This is not too surprising since LLVM is much larger and sophisticated - it has a more powerful register allocator, supports things like float32, etc. Those two examples weren't random - Firefox has added an LLVM-like register allocator and float32 support (not just for asm.js code, in general). So there are two paths here: Safari takes LLVM, and Firefox implements some bits from LLVM in its own JIT. v8 so far has done neither, but hopefully has plans to do one of the two, or some other cool thing. In addition there is the issue of asm.js. Safari doesn't detect it, and beats Chrome but still trails Firefox. But it's far from clear that is the last word - Safari's new JIT just landed, it can be tuned a lot more, presumably. So it might still end up showing that a general JIT can beat custom asm.js optimizations, time will tell. What is clear though is that the Safari's FTL JIT took far more work than Firefox's asm.js optimizations. If the LLVM-based JIT mostly helps asm.js code anyhow - it shows little benefits in other benchmarks so far - then that raises interesting questions (if the benefit is mostly for asm.js, why not focus on asm.js specifically, if it's simpler and more effective?). But again, FTL just landed, perhaps with more tuning it will show general benefits. Exciting times!

flohofwoe 12y ago

IIRC, V8 had implemented optimizations that specifically target asm.js code patterns in the past (the performance gap for asm.js code between FF and Chrome definitely got a lot smaller in 2013), they just don't rely on the "use asm" keyword or do AOT compilation like Odinmonkey.

lucian1900 12y ago
It was more fixing bugs in their JIT that were only really exercised by asm.js-like code.

sanxiyn 12y ago

Edit: the particular quote I objected to was edited out by parent. I object to "all the dynamicness of the rest of the JS is what V8 excels at". V8 excels at a subset of dynamic JavaScript (namely those benefit from Crankshaft) and is not very good outside of the subset. Typescript benchmark included in Octane benchmark suite is a good example. In Google's own words: "This test measures the time TypeScript takes to compile itself and is a proxy of how well a VM handles complex and sizable Javascript applications". In particular, Crankshaft is not effective for this benchmark. (Check yourself by running it with Crankshaft disabled: the score actually improves because you save compilation effort.) And as you can check yourself or check on AreWeFastYet, JavaScriptCore beats V8 by 2x on this benchmark. I admit Crankshaft subset of JavaScript is large and interesting (certainly compared to asm.js subset) but it is not true that V8 excels at dynamic JavaScript, for sufficiently large value of dynamic. JavaScriptCore does.

mantrax5 12y ago
Noted. Sorry for removing this, I indeed said that, but I thought it's redundant so I removed it.
mraleph 12y ago
TypeScript benchmark is one of the Octane benchmarks that does not do warm up, so Crankshaft does not have chance to fully shine [which is totally fair given how TypeScript compiler is used: you don't warm it up, you just spawn it again and again]. It also has an extremely flat profile. So you can't simply say that "Crankshaft is not effective for it". Though potentially you can say "benefits of Crankshafting do not compensate for the cost of Crankshafting". If you enable warmup for it then score jumps by a factor of 2 for V8. JSC might be warming faster or simply have a better cost / benefit ratio at lower tiers. I doubt FTL kicks at all for this benchmark given how short running it is. V8 also deopts quite a bit on it, so it might be some pathological case... This is definitely something worth exploring and fixing on V8 side.
zurn 12y ago
> V8 excels at a subset of dynamic JavaScript (namely those benefit from Crankshaft) and is not very good outside of the subset As I understand it Crankshaft's claim to fame is the runtime type specialization, which does well if the code is hot and ends up using same types in all cases. Is this what you are referring to?

scriptproof 12y ago

Your comment contradicts itself: "Asm.js is a JS subset" and "V8 isn't implementing Asm.js in any way.". It is fully implemented, of course, it is only the "use asm.js" directive that is not. That prevents some control to signal non asm.js code in a JS file, and only that.

azakai 12y ago

I don't think there is a "victory" here. Just an important milestone for Safari/JavaScriptCore. They have done some amazing engineering here! In the broader context, this shows that the JSC approach of taking LLVM and using it as a JIT - called FTL - can beat Chrome/v8 on large long-running computational code. That means that LLVM is significantly better than v8's backend JIT at some things. This is not too surprising since LLVM is much larger and sophisticated - it has a more powerful register allocator, supports things like float32, etc. Those two examples weren't random - Firefox has added an LLVM-like register allocator and float32 support (not just for asm.js code, in general). So there are two paths here: Safari takes LLVM, and Firefox implements some bits from LLVM in its own JIT. v8 so far has done neither, but hopefully has plans to do one of the two, or some other cool thing. In addition there is the issue of asm.js. Safari doesn't detect it, and beats Chrome but still trails Firefox. But it's far from clear that is the last word - Safari's new JIT just landed, it can be tuned a lot more, presumably. So it might still end up showing that a general JIT can beat custom asm.js optimizations, time will tell. What is clear though is that the Safari's FTL JIT took far more work than Firefox's asm.js optimizations. If the LLVM-based JIT mostly helps asm.js code anyhow - it shows little benefits in other benchmarks so far - then that raises interesting questions (if the benefit is mostly for asm.js, why not focus on asm.js specifically, if it's simpler and more effective?). But again, FTL just landed, perhaps with more tuning it will show general benefits. Exciting times!