4 ms·

In case you're curious: http://arewefastyet.com/ http://arewefastyet.com/ https://areweslimyet.com/ https://areweslimyet.com/

by aroch 13y ago

twoodfin 13y ago
Any idea why the "slim" numbers are moving steadily in the wrong direction?
- agapos 13y ago
  Changes in browser engine and method/elaboration in memory measurement. And possibly a lot of regressions... Problem is, this does not convince Average Joe about the slimness of Fx. I asked them to set up a main chart using only the stable ones with the latest changes (allocation), but it was ignored. IMHO that would help them see the what the end user would see over the long course, and A. Joe would see what actually effects him (as he is unlikely to use dev channels). Edit: the charts also do not show the improvements in Garbage Collection (IGC in Fx16, upcoming ICC and GGC) and the memory management with add-os (Hueyfix in Fx15).
- mccr8 13y ago
  Tiny regressions are hard to track down, but over time they add up.
- woogle 13y ago
  I'm wondering too. FF13 seems the best with "slim" numbers.
- aroch 13y ago
  Some of it is the move from single to multi process/thread tab and rendering model. And if I'm remembering correctly a few of the new HTML5 elements added have caused significant (+20-30mb) leaks as they are, as of yet, unoptimized in the rendering code.
  - tedmielczarek 13y ago
    There's no multiprocess content code enabled by default yet, so that's not it.
    - aroch 13y ago
      Not process but I could hsave sworn there are multithreaded processes enabled by default in Aurora, image processing for example.
      - sp332 13y ago
        It's enabled by default in version 30, which is Aurora right now.
- tedmielczarek 13y ago
  Controlling memory usage is a battle that requires constant vigilance, and it's hard. It's difficult to measure memory usage in a reliable, repeatable way such that we could use it to prevent regressions. Additionally, small regressions can add up over time, making it hard to pinpoint any one thing that could be backed out to improve memory usage.
- nnethercote 13y ago
  Partly it's just that the browser is getting more features. Things like WebRTC are a ton of code. More generally, AWSY is useful for identifying some regressions, but it's one specific, unrealistic workload. You can't use it to judge Firefox's overall memory consumption. For example, looking at AWSY you'd think Firefox 13 had the best memory consumption. But I guarantee you that Firefox 28 has better memory consumption. In particular, Firefox 28 has fewer bad cases where memory consumption spirals out of control, and those are the cases that really hurt performance and stability. Using 10 or 20% more memory at start-up, for example, isn't a big deal in comparison.
  - nnethercote 13y ago
    Nonetheless, the increase isn't good. I filed https://bugzilla.mozilla.org/show_bug.cgi?id=986323 https://bugzilla.mozilla.org/show_bug.cgi?id=986323 about this, and have started investigating.
LukeG 13y ago
Anyone have a good breakdown of the differences in these benchmarks? i.e. Why does Chrome dramatically lead the(ir) Octane benchmark?
- Avshalom 13y ago
  Because Octane was written by Google. So it's either: -representative of things Google knows it's faster at -representative of things Google thinks should be fast and prioritizes... and is thus faster. or a little of column A little of column b in a feedback loop.
  - gsnedders 13y ago
    Octane is the continuation of V8 benchmark (and contains all of the V8 benchmark tests) — V8 (the JS engine) was specifically designed to provide good performance on those tests (this is why V8 had a generational GC from launch, for example; though somewhat amusing last I checked Carakan (Opera/Presto's last JS engine) was quickest at splay, primarily a GC benchmark, despite having a very naïve GC!). The tests new in Octane don't display such a clear disparity, as V8 wasn't designed for them.
    - mraleph 13y ago
      hint: Splay is a benchmark that penalizes generational GCs. If you have a fast mark-sweep GC with an aggressive growth policy you are going to totally rock it.
      - gsnedders 13y ago
        I may slightly be showing how long it's been since I looked at Splay. :) My memory was that originally V8 was out-performing everyone else primarily because the initial allocation (into the nursery) was way quicker than anyone else did. Or maybe it was freeing the nodes that die young? Certainly I remember both SpiderMonkey and Carakan spending huge amounts of time on GC on Splay, and Carakan's massive speed-up came when the object representation was changed (GC actually asymptotically regressed as a result, but in almost every practical case cache-locality (and reduction of memory accesses) outweighed it).

In case you're curious: http://arewefastyet.com/ http://arewefastyet.com/ https://areweslimyet.com/ https://areweslimyet.com/

by aroch 13y ago

In case you're curious: http://arewefastyet.com/ http://arewefastyet.com/ https://areweslimyet.com/ https://areweslimyet.com/

twoodfin 13y ago

Any idea why the "slim" numbers are moving steadily in the wrong direction?

agapos 13y ago
Changes in browser engine and method/elaboration in memory measurement. And possibly a lot of regressions... Problem is, this does not convince Average Joe about the slimness of Fx. I asked them to set up a main chart using only the stable ones with the latest changes (allocation), but it was ignored. IMHO that would help them see the what the end user would see over the long course, and A. Joe would see what actually effects him (as he is unlikely to use dev channels). Edit: the charts also do not show the improvements in Garbage Collection (IGC in Fx16, upcoming ICC and GGC) and the memory management with add-os (Hueyfix in Fx15).
mccr8 13y ago
Tiny regressions are hard to track down, but over time they add up.
woogle 13y ago
I'm wondering too. FF13 seems the best with "slim" numbers.
aroch 13y ago
Some of it is the move from single to multi process/thread tab and rendering model. And if I'm remembering correctly a few of the new HTML5 elements added have caused significant (+20-30mb) leaks as they are, as of yet, unoptimized in the rendering code.
- tedmielczarek 13y ago
  There's no multiprocess content code enabled by default yet, so that's not it.
  - aroch 13y ago
    Not process but I could hsave sworn there are multithreaded processes enabled by default in Aurora, image processing for example.
    - sp332 13y ago
      It's enabled by default in version 30, which is Aurora right now.
tedmielczarek 13y ago
Controlling memory usage is a battle that requires constant vigilance, and it's hard. It's difficult to measure memory usage in a reliable, repeatable way such that we could use it to prevent regressions. Additionally, small regressions can add up over time, making it hard to pinpoint any one thing that could be backed out to improve memory usage.
nnethercote 13y ago
Partly it's just that the browser is getting more features. Things like WebRTC are a ton of code. More generally, AWSY is useful for identifying some regressions, but it's one specific, unrealistic workload. You can't use it to judge Firefox's overall memory consumption. For example, looking at AWSY you'd think Firefox 13 had the best memory consumption. But I guarantee you that Firefox 28 has better memory consumption. In particular, Firefox 28 has fewer bad cases where memory consumption spirals out of control, and those are the cases that really hurt performance and stability. Using 10 or 20% more memory at start-up, for example, isn't a big deal in comparison.
- nnethercote 13y ago
  Nonetheless, the increase isn't good. I filed https://bugzilla.mozilla.org/show_bug.cgi?id=986323 https://bugzilla.mozilla.org/show_bug.cgi?id=986323 about this, and have started investigating.

LukeG 13y ago

Anyone have a good breakdown of the differences in these benchmarks? i.e. Why does Chrome dramatically lead the(ir) Octane benchmark?

Avshalom 13y ago
Because Octane was written by Google. So it's either: -representative of things Google knows it's faster at -representative of things Google thinks should be fast and prioritizes... and is thus faster. or a little of column A little of column b in a feedback loop.
- gsnedders 13y ago
  Octane is the continuation of V8 benchmark (and contains all of the V8 benchmark tests) — V8 (the JS engine) was specifically designed to provide good performance on those tests (this is why V8 had a generational GC from launch, for example; though somewhat amusing last I checked Carakan (Opera/Presto's last JS engine) was quickest at splay, primarily a GC benchmark, despite having a very naïve GC!). The tests new in Octane don't display such a clear disparity, as V8 wasn't designed for them.
  - mraleph 13y ago
    hint: Splay is a benchmark that penalizes generational GCs. If you have a fast mark-sweep GC with an aggressive growth policy you are going to totally rock it.
    - gsnedders 13y ago
      I may slightly be showing how long it's been since I looked at Splay. :) My memory was that originally V8 was out-performing everyone else primarily because the initial allocation (into the nursery) was way quicker than anyone else did. Or maybe it was freeing the nodes that die young? Certainly I remember both SpiderMonkey and Carakan spending huge amounts of time on GC on Splay, and Carakan's massive speed-up came when the object representation was changed (GC actually asymptotically regressed as a result, but in almost every practical case cache-locality (and reduction of memory accesses) outweighed it).