7 ms·

Firefox’s New Memory Tool

251 points by bpierre 11y ago

aembleton 11y ago
This looks good. Will it be made available in the regular Firefox Inspector Element tool any time soon?
- callahad 11y ago
  It should land in the Release channel in 12 weeks (things move from Developer Edition to Beta to Release every six weeks)
- deleted 11y ago
  [deleted]
sehr 11y ago
Is this the part of devtools that was built using React & Redux? I remember hearing something a it that a few weeks ago
- mnemonik 11y ago
  Yes it is. I also wrote more about the backend implementation on lobste.rs <https://lobste.rs/s/cvtz0x/firefox_s_new_memory_tool https://lobste.rs/s/cvtz0x/firefox_s_new_memory_tool >: Hi folks, For the last year or so, my coworker Jim Blandy and I have been designing and implementing APIs in SpiderMonkey (Firefox’s JavaScript engine) to support various kinds of memory allocation/retention introspection. We are finally shipping a frontend to users as well, and this blog post is an intro to that. This is very much the tip of the iceberg and we wanted to just start shipping since we’ve been building APIs for so long, but this is really a fraction of the things planned. Nightly already has filtering/searching and will soon have import/export of snapshots as well as diffing <http://i.imgur.com/qjWOERs.gif http://i.imgur.com/qjWOERs.gif >. Dominator trees and shortest paths from GC roots are incoming as well. How is this built? We made an abstraction called “ubi::Node” <https://dxr.mozilla.org/mozilla-central/rev/a8ed7dd831d1969a5a1a8636e63bd93d6aeaf94a/js/public/UbiNode.h#31-163 https://dxr.mozilla.org/mozilla-central/rev/a8ed7dd831d1969a... > (for “ubiquitous node”) that sees the GC’s heap graph with a traditional nodes-and-edges view so that we don’t have to worry about the difference between (say) an object and some jit code when writing graph analyses. What is cool is that this interface can be backed by either the live heap graph or an offline heap graph. When you save a heap snapshot, we traverse this graph and serialize it into protobuf messages. These can then be reconstructed into an offline graph which you can run analyses on at your leisure. We also make pretty extensive use of stack capturing, because that is a concrete way to tie retention and GC pressure back to something that feels concrete to users: locations in their source code. I’ve written a bit about how we minimize the overhead of (potentially very frequent) stack captures before by hash consing individual frames so that tails are shared <http://fitzgeraldnick.com/weblog/61/ http://fitzgeraldnick.com/weblog/61/ >. Since then, we have also found a neat way to avoid walking the stack repeatedly if we have already done so once. If anyone has any questions, please ask away :) And I made a quick update: Also, Jim loved writing this comment so much (it is a fun read) that I feel I have to share it :) https://dxr.mozilla.org/mozilla-central/rev/a8ed7dd831d1969a5a1a8636e63bd93d6aeaf94a/mfbt/FastBernoulliTrial.h#18-172 https://dxr.mozilla.org/mozilla-central/rev/a8ed7dd831d1969a... We use this for sampling object allocations to give insight into GC pressure.
  - sams99 11y ago
    awesome tool, I HUGE help would be allowing us to diff snapshots to catch leaks, I can see something is leaking but can not exactly track down what it is.
    - mnemonik 11y ago
      Yup, a patch is in review right now: https://bugzilla.mozilla.org/show_bug.cgi?id=960780 https://bugzilla.mozilla.org/show_bug.cgi?id=960780 Not sure if you missed it in the parent post, but here is a sneak peek: http://i.imgur.com/qjWOERs.gif http://i.imgur.com/qjWOERs.gif
- mbrubeck 11y ago
  Yes. You can see the code here: https://dxr.mozilla.org/mozilla-central/source/devtools/client/memory https://dxr.mozilla.org/mozilla-central/source/devtools/clie... (I think there are other parts of the developer tools that also use React and Redux, like the debugger.)
  - jlongster 11y ago
    Not quite yet, but I'm very close to landing a debugger refactor which uses redux internally :) https://bugzilla.mozilla.org/show_bug.cgi?id=1200798 https://bugzilla.mozilla.org/show_bug.cgi?id=1200798 And we're going to slow introduce React.
ZenoArrow 11y ago
This looks like a useful tool, thank you to the team at Mozilla responsible for this new memory tool. Also, when watching the introductory video I was surprised that disabling tracking scripts offered so much potential for reducing memory usage (depending on the site of course). Is this new feature only available in private browsing mode, or can it be enabled for standard browsing? Are there any downsides for the user when disabling tracking scripts all the time?
- mnemonik 11y ago
  Yes, it can be enabled all the time with about:config -> privacy.trackingprotection.enabled = true
- rockdoe 11y ago
  >Are there any downsides for the user when disabling tracking scripts all the time? Some websites break. In my experience very few though. Also many seem to be missing a lot of ads. This is terrible, cough cough.
- cyber 11y ago
  For a look into how disabling/enabling privacy trackers impact sites, try out the Privacy Badger plugin. It's got some granular support, and crowdsourced guidance. With a button to report when it broke a site! Privacy Badger is an EFF project.
- JaRail 11y ago
  Really? You're surprised? Third party advertisers are a plague to good engineers.. Same as turning on adblock, everything runs so much cleaner.
EdSharkey 11y ago
This looks so advanced, Firefox is the best! I'm very curious to watch how WeakMap behaves and how it is visualized in this tool! Speaking of WeakMap, perhaps a Firefox engineer could satisfy my curiosity on this subject. What is the algorithm Firefox uses to release weakly held references? Or, perhaps better restated: how aggressive is Firefox in holding on to weakly referenced objects? When the majority of new JS memory allocations on a page get placed into WeakMaps, does Firefox always continue to allocate memory from the host OS, or is there some threshold where it starts to let things slip? Also, is there any rhyme or reason to what gets released first from WeakMaps? Can Firefox run wild like it does with strongly held references, allocating into WeakMaps until the host OS cannot satisfy new native memory allocation requests and only then start to release weak references? Also, how crazy do things get for you when you put lots of DOM objects or, even worse, GPU-backed memory structures like CanvasPattern into a WeakMap? Please share cool war stories! ;) Disclaimer: I have not used WeakMap in any serious capacity, so I don't know what I'm talking about.
- mnemonik 11y ago
  Up until a couple months ago, it had an O(n^2) marking algorithm that kept iterating until it got to a fixed point. Thanks to the hard work of Steve Fink, these days it has an O(n) marking algorithm. See https://bugzilla.mozilla.org/show_bug.cgi?id=1164294 https://bugzilla.mozilla.org/show_bug.cgi?id=1164294 for the details. Edit to address your questions more directly: > how aggressive is Firefox in holding on to weakly referenced objects? They are just collected at the next GC slice where they are no longer retained. SpiderMonkey doesn't do anything to try and eagerly collect them. GC slices generally won't happen until either there has been enough pressure (allocations) to trigger one or the user is inactive for N amount of time. There are tons of GC triggering heuristics and some are probably not optimal. It is on the GC team's TODO list to revisit and clean these up. > does Firefox always continue to allocate memory from the host OS, or is there some threshold where it starts to let things slip? The threshold of how much is allocated before triggering GC grows as the retained set's size grows. This is bounded by available memory. Aaaaand I have a meeting I need to go attend.
  - EdSharkey 11y ago
robotkilla 11y ago
Super excited to see this released - perfect timing as I've been working on JS heavily as of late and have been searching for a good memory debugging tool and I'm not too big on chrome's option. This looks easier to read and use overall.
solidr53 11y ago
Chrome has had that for at least 2 years now. Sure, Firefox has some additions, but how is Firefox doing some breakthrough? https://developer.chrome.com/devtools/docs/javascript-memory-profiling https://developer.chrome.com/devtools/docs/javascript-memory...
- ikawe 11y ago
  I wish you'd asked it like this: Chrome has had memory profiling for at least 2 years. https://developer.chrome.com/devtools/docs/javascript-memory-profiling https://developer.chrome.com/devtools/docs/javascript-memory... How does Firefox's new profiling feature compare?
  - kohlerm 11y ago
    Chrome's tool supports a dominator tree view. Sorry, also I'm excited to see a memory usage tool for firefox is now available. Supporting a dominator tree view is IMHO a crucial feature you will need sooner or later for investigating memory usage issues. The predecessor to the Eclipse Memory Analyzer had that in 2006/2007.
    - jsantell 11y ago
      The dominator tree view[0] is one of the first things being worked on after release. [0] https://bugzilla.mozilla.org/show_bug.cgi?id=1149367 https://bugzilla.mozilla.org/show_bug.cgi?id=1149367
jscheel 11y ago
I'm really impressed with the new firefox dev tools. I'm still getting several crashes a day with them though, so maybe this can help internally too :) All joking aside, this is a great addition.
- mnemonik 11y ago
  Can you file bug(s) with links to the crash report(s) in about:crashes? Alternatively, you could email me your crash reports (email in profile) and I could file bug(s) for youe.
kevingadd 11y ago
This looks amazing, can't wait to see it motivate the v8/blink people to deliver an equivalent. Storing allocation stacks is essential for troubleshooting leaks and I've never been able to get anything remotely close to that on the web.
eveningcoffee 11y ago
This is very cool. I wish there is also kind of tool that shows CPU, memory and network usage for every open page in one place.

251 points by bpierre 11y ago

aembleton 11y ago

This looks good. Will it be made available in the regular Firefox Inspector Element tool any time soon?

callahad 11y ago
It should land in the Release channel in 12 weeks (things move from Developer Edition to Beta to Release every six weeks)
deleted 11y ago
[deleted]

sehr 11y ago

Is this the part of devtools that was built using React & Redux? I remember hearing something a it that a few weeks ago

mnemonik 11y ago
Yes it is. I also wrote more about the backend implementation on lobste.rs <https://lobste.rs/s/cvtz0x/firefox_s_new_memory_tool https://lobste.rs/s/cvtz0x/firefox_s_new_memory_tool >: Hi folks, For the last year or so, my coworker Jim Blandy and I have been designing and implementing APIs in SpiderMonkey (Firefox’s JavaScript engine) to support various kinds of memory allocation/retention introspection. We are finally shipping a frontend to users as well, and this blog post is an intro to that. This is very much the tip of the iceberg and we wanted to just start shipping since we’ve been building APIs for so long, but this is really a fraction of the things planned. Nightly already has filtering/searching and will soon have import/export of snapshots as well as diffing <http://i.imgur.com/qjWOERs.gif http://i.imgur.com/qjWOERs.gif >. Dominator trees and shortest paths from GC roots are incoming as well. How is this built? We made an abstraction called “ubi::Node” <https://dxr.mozilla.org/mozilla-central/rev/a8ed7dd831d1969a5a1a8636e63bd93d6aeaf94a/js/public/UbiNode.h#31-163 https://dxr.mozilla.org/mozilla-central/rev/a8ed7dd831d1969a... > (for “ubiquitous node”) that sees the GC’s heap graph with a traditional nodes-and-edges view so that we don’t have to worry about the difference between (say) an object and some jit code when writing graph analyses. What is cool is that this interface can be backed by either the live heap graph or an offline heap graph. When you save a heap snapshot, we traverse this graph and serialize it into protobuf messages. These can then be reconstructed into an offline graph which you can run analyses on at your leisure. We also make pretty extensive use of stack capturing, because that is a concrete way to tie retention and GC pressure back to something that feels concrete to users: locations in their source code. I’ve written a bit about how we minimize the overhead of (potentially very frequent) stack captures before by hash consing individual frames so that tails are shared <http://fitzgeraldnick.com/weblog/61/ http://fitzgeraldnick.com/weblog/61/ >. Since then, we have also found a neat way to avoid walking the stack repeatedly if we have already done so once. If anyone has any questions, please ask away :) And I made a quick update: Also, Jim loved writing this comment so much (it is a fun read) that I feel I have to share it :) https://dxr.mozilla.org/mozilla-central/rev/a8ed7dd831d1969a5a1a8636e63bd93d6aeaf94a/mfbt/FastBernoulliTrial.h#18-172 https://dxr.mozilla.org/mozilla-central/rev/a8ed7dd831d1969a... We use this for sampling object allocations to give insight into GC pressure.
- sams99 11y ago
  awesome tool, I HUGE help would be allowing us to diff snapshots to catch leaks, I can see something is leaking but can not exactly track down what it is.
  - mnemonik 11y ago
    Yup, a patch is in review right now: https://bugzilla.mozilla.org/show_bug.cgi?id=960780 https://bugzilla.mozilla.org/show_bug.cgi?id=960780 Not sure if you missed it in the parent post, but here is a sneak peek: http://i.imgur.com/qjWOERs.gif http://i.imgur.com/qjWOERs.gif
mbrubeck 11y ago
Yes. You can see the code here: https://dxr.mozilla.org/mozilla-central/source/devtools/client/memory https://dxr.mozilla.org/mozilla-central/source/devtools/clie... (I think there are other parts of the developer tools that also use React and Redux, like the debugger.)
- jlongster 11y ago
  Not quite yet, but I'm very close to landing a debugger refactor which uses redux internally :) https://bugzilla.mozilla.org/show_bug.cgi?id=1200798 https://bugzilla.mozilla.org/show_bug.cgi?id=1200798 And we're going to slow introduce React.

ZenoArrow 11y ago

This looks like a useful tool, thank you to the team at Mozilla responsible for this new memory tool. Also, when watching the introductory video I was surprised that disabling tracking scripts offered so much potential for reducing memory usage (depending on the site of course). Is this new feature only available in private browsing mode, or can it be enabled for standard browsing? Are there any downsides for the user when disabling tracking scripts all the time?

mnemonik 11y ago
Yes, it can be enabled all the time with about:config -> privacy.trackingprotection.enabled = true
rockdoe 11y ago
>Are there any downsides for the user when disabling tracking scripts all the time? Some websites break. In my experience very few though. Also many seem to be missing a lot of ads. This is terrible, cough cough.
cyber 11y ago
For a look into how disabling/enabling privacy trackers impact sites, try out the Privacy Badger plugin. It's got some granular support, and crowdsourced guidance. With a button to report when it broke a site! Privacy Badger is an EFF project.
JaRail 11y ago
Really? You're surprised? Third party advertisers are a plague to good engineers.. Same as turning on adblock, everything runs so much cleaner.

EdSharkey 11y ago

This looks so advanced, Firefox is the best! I'm very curious to watch how WeakMap behaves and how it is visualized in this tool! Speaking of WeakMap, perhaps a Firefox engineer could satisfy my curiosity on this subject. What is the algorithm Firefox uses to release weakly held references? Or, perhaps better restated: how aggressive is Firefox in holding on to weakly referenced objects? When the majority of new JS memory allocations on a page get placed into WeakMaps, does Firefox always continue to allocate memory from the host OS, or is there some threshold where it starts to let things slip? Also, is there any rhyme or reason to what gets released first from WeakMaps? Can Firefox run wild like it does with strongly held references, allocating into WeakMaps until the host OS cannot satisfy new native memory allocation requests and only then start to release weak references? Also, how crazy do things get for you when you put lots of DOM objects or, even worse, GPU-backed memory structures like CanvasPattern into a WeakMap? Please share cool war stories! ;) Disclaimer: I have not used WeakMap in any serious capacity, so I don't know what I'm talking about.

mnemonik 11y ago
Up until a couple months ago, it had an O(n^2) marking algorithm that kept iterating until it got to a fixed point. Thanks to the hard work of Steve Fink, these days it has an O(n) marking algorithm. See https://bugzilla.mozilla.org/show_bug.cgi?id=1164294 https://bugzilla.mozilla.org/show_bug.cgi?id=1164294 for the details. Edit to address your questions more directly: > how aggressive is Firefox in holding on to weakly referenced objects? They are just collected at the next GC slice where they are no longer retained. SpiderMonkey doesn't do anything to try and eagerly collect them. GC slices generally won't happen until either there has been enough pressure (allocations) to trigger one or the user is inactive for N amount of time. There are tons of GC triggering heuristics and some are probably not optimal. It is on the GC team's TODO list to revisit and clean these up. > does Firefox always continue to allocate memory from the host OS, or is there some threshold where it starts to let things slip? The threshold of how much is allocated before triggering GC grows as the retained set's size grows. This is bounded by available memory. Aaaaand I have a meeting I need to go attend.
- EdSharkey 11y ago

robotkilla 11y ago

Super excited to see this released - perfect timing as I've been working on JS heavily as of late and have been searching for a good memory debugging tool and I'm not too big on chrome's option. This looks easier to read and use overall.

solidr53 11y ago

Chrome has had that for at least 2 years now. Sure, Firefox has some additions, but how is Firefox doing some breakthrough? https://developer.chrome.com/devtools/docs/javascript-memory-profiling https://developer.chrome.com/devtools/docs/javascript-memory...

ikawe 11y ago
I wish you'd asked it like this: Chrome has had memory profiling for at least 2 years. https://developer.chrome.com/devtools/docs/javascript-memory-profiling https://developer.chrome.com/devtools/docs/javascript-memory... How does Firefox's new profiling feature compare?
- kohlerm 11y ago
  Chrome's tool supports a dominator tree view. Sorry, also I'm excited to see a memory usage tool for firefox is now available. Supporting a dominator tree view is IMHO a crucial feature you will need sooner or later for investigating memory usage issues. The predecessor to the Eclipse Memory Analyzer had that in 2006/2007.
  - jsantell 11y ago
    The dominator tree view[0] is one of the first things being worked on after release. [0] https://bugzilla.mozilla.org/show_bug.cgi?id=1149367 https://bugzilla.mozilla.org/show_bug.cgi?id=1149367

jscheel 11y ago

I'm really impressed with the new firefox dev tools. I'm still getting several crashes a day with them though, so maybe this can help internally too :) All joking aside, this is a great addition.

mnemonik 11y ago
Can you file bug(s) with links to the crash report(s) in about:crashes? Alternatively, you could email me your crash reports (email in profile) and I could file bug(s) for youe.

kevingadd 11y ago

This looks amazing, can't wait to see it motivate the v8/blink people to deliver an equivalent. Storing allocation stacks is essential for troubleshooting leaks and I've never been able to get anything remotely close to that on the web.

eveningcoffee 11y ago

This is very cool. I wish there is also kind of tool that shows CPU, memory and network usage for every open page in one place.