4 ms·

Benchmarking Firefox 83 Nightly with “Warp” Against Google Chrome on Linux

33 points by electricant 6y ago

skavi 6y ago
Has any decent benchmarking been done on the recently released Safari 14? Apple was promising some fairly significant performance improvements. I ran Speedometer 2.0 before and after the iOS 14 update and saw the score increase from 158 to 168. Unfortunately, I did not think to do the same for the macOS update.
bscphil 6y ago
Wow. As a non-expert about any of this technology, I must say that these comparisons are surprising. Maybe even too surprising. I wonder if maybe this could be the result of Firefox being built with different options for the nightly build than the release build. That might help explain the difference. Then again, if I'm not mistaken I recall some messaging from Mozilla about Warp improving render time of heavy webapps without really helping the objective benchmarks. With such a consistent set of results it's hard to see how that could be the case (unless the benchmarks are once again behind the times), but I'm willing to give the benefit of the doubt on this. The worst case is, if it proves to be a regression, it'll probably get reverted. And Firefox as-is is already fast enough for my needs.
- slaymaker1907 6y ago
  Maybe they are optimizing stuff like compile times at the cost of throughout? My guess is that most JS code either doesn't need JIT at all or only needs the most minimal JIT and benchmarks tend to be bad at measuring those cases.
throwaway77384 6y ago
"Google Chrome is still delivering the leading performance on Linux." Yup, same here. I use FF for everything, but occasionally have to use Chrome for something. Whenever I do, the performance difference is noticeable. But I ignore it, because I use FF out of principle. These new JIT speed-ups paired with the potential for properly hardware accelerated video on Linux make me hopeful for future discussions with people who will (unfortunately, so far, rightfully) claim "but Chrome is faster", which you can't argue against....
jiripospisil 6y ago
I don't think benchmarking a Nightly release is valuable. The code literally changes every day, things get slower only to get faster a few days later (or vice versa). For example, the article mentions a regression in the JetStream 2.0 benchmark. There's a ticket about this that was fixed just a few hours ago [0]. There are other performance related tickets currently open (or recently closed) [1] that should make the benchmark's results no longer accurate. [0] https://bugzilla.mozilla.org/show_bug.cgi?id=1667864 https://bugzilla.mozilla.org/show_bug.cgi?id=1667864 [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1613592 https://bugzilla.mozilla.org/show_bug.cgi?id=1613592 (look at the top for "Depends on" tickets)

33 points by electricant 6y ago

Has any decent benchmarking been done on the recently released Safari 14? Apple was promising some fairly significant performance improvements. I ran Speedometer 2.0 before and after the iOS 14 update and saw the score increase from 158 to 168. Unfortunately, I did not think to do the same for the macOS update.

bscphil 6y ago

Wow. As a non-expert about any of this technology, I must say that these comparisons are surprising. Maybe even too surprising. I wonder if maybe this could be the result of Firefox being built with different options for the nightly build than the release build. That might help explain the difference. Then again, if I'm not mistaken I recall some messaging from Mozilla about Warp improving render time of heavy webapps without really helping the objective benchmarks. With such a consistent set of results it's hard to see how that could be the case (unless the benchmarks are once again behind the times), but I'm willing to give the benefit of the doubt on this. The worst case is, if it proves to be a regression, it'll probably get reverted. And Firefox as-is is already fast enough for my needs.

slaymaker1907 6y ago
Maybe they are optimizing stuff like compile times at the cost of throughout? My guess is that most JS code either doesn't need JIT at all or only needs the most minimal JIT and benchmarks tend to be bad at measuring those cases.

throwaway77384 6y ago

"Google Chrome is still delivering the leading performance on Linux." Yup, same here. I use FF for everything, but occasionally have to use Chrome for something. Whenever I do, the performance difference is noticeable. But I ignore it, because I use FF out of principle. These new JIT speed-ups paired with the potential for properly hardware accelerated video on Linux make me hopeful for future discussions with people who will (unfortunately, so far, rightfully) claim "but Chrome is faster", which you can't argue against....

jiripospisil 6y ago

I don't think benchmarking a Nightly release is valuable. The code literally changes every day, things get slower only to get faster a few days later (or vice versa). For example, the article mentions a regression in the JetStream 2.0 benchmark. There's a ticket about this that was fixed just a few hours ago [0]. There are other performance related tickets currently open (or recently closed) [1] that should make the benchmark's results no longer accurate. [0] https://bugzilla.mozilla.org/show_bug.cgi?id=1667864 https://bugzilla.mozilla.org/show_bug.cgi?id=1667864 [1] https://bugzilla.mozilla.org/show_bug.cgi?id=1613592 https://bugzilla.mozilla.org/show_bug.cgi?id=1613592 (look at the top for "Depends on" tickets)