4 ms·

Modern operating systems will grant large amounts of RAM to web browsers in general to improve caching. That RAM is easily relinquished back to the OS if anothe

Modern operating systems will grant large amounts of RAM to web browsers in general to improve caching. That RAM is easily relinquished back to the OS if another app needs it. Was the 12GB with another RAM-intensive app running? Or just Firefox by itself.

0xdead 7y ago
That's not true. Operating systems don't know if a process is a browser.
- basch 7y ago
  OS > Hey I need RAM Browser > OK we can purge these tabs first is how it should work. If the first step doesnt happen, its not clear if the second one should execute by itself. Id be much more concerned about background CPU usage. Eats battery, and actually slows the computer down. That all said, its pretty clear that browsers are leaking RAM sometimes. I remember bed bath and beyond using 9 or 12GB of ram the other month.
  - blattimwind 7y ago
    > is how it should work. That's not how it works. Browser might monitor free system memory actively and release memory if the system runs low, but at least Firefox doesn't do that. There is no mechanism (in Windows, Linux) that allows the OS to notify applications of memory pressure or that allows the application to tell the OS that some allocations can be thrown out if need be. (Windows has a mechanism - MEM_RESET - that only works with paged out allocations) Edit: Windows 10 (8?) actually expands on MEM_RESET with Offer/ReclaimVirtualMemory, but I kinda doubt browsers use this, because these have similar semantics to MEM_RESET (except they seem to work without paging, which is good). For an application that wants to cache not-well-defined amounts of data itself these might be very useful though, because you can make the data structure simple enough that it works with these.
    - zozbot234 7y ago
      > ...that allows the OS to notify applications of memory pressure or that allows the application to tell the OS that some allocations can be thrown out if need be. There are mechanisms for both of these in up-to-date Linux - pressure stall information and volatile ranges, respectively.
      - blattimwind 7y ago
        MADV_FREE didn't work properly until fairly recently, though, because it behaved like Windows' MEM_RESET. AIUI MADV_FREE is not intended for ranges that you actually want to work with (i.e. you want to keep the data, not just the virtual addresses), but rather as an optimization for allocators.
    - mauricioc 7y ago
      https://dblohm7.ca/blog/2015/07/28/interesting-win32-apis/ https://dblohm7.ca/blog/2015/07/28/interesting-win32-apis/ notes that Windows has the required APIs, and that Firefox already supports handling memory pressure notifications on non-Windows platforms. Not sure how up-to-date this information is.
    - kevingadd 7y ago
      Firefox does have a mechanism for reducing memory pressure, there's a button for it in about:memory. I can't guarantee that this flow actually happens upon low memory states on Windows etc but it wouldn't be hard to at least hack it together. Monitoring available memory (say, on a timer) is not a difficult thing to do on Windows - it would just be difficult to make it completely robust.
      - blattimwind 7y ago
        Well it definitely doesn't work on Linux, at least not automatically.
        kevingadd 7y ago
        It's important to specify your test scenario here and the behavior you expect, as it's a known issue in general that Linux behaves poorly under OOM conditions. It's not necessarily Firefox specific. If all of my memory is "in use" but half of it is cached file pages, should Firefox compact its heap and evict cached images? What if all of my memory is spoken for but a bunch of it isn't actually committed, due to overcommit?
  - glandium 7y ago
    Another strategy is: Browser> So this memory here, I don't really care if you swipe it in the future. (later) OS> Yum yum (later) Browser> Oh, that buffer is empty, let me fill it again. Firefox uses it, IIRC, for decoded image buffers. https://searchfox.org/mozilla-central/source/memory/volatile/VolatileBuffer.h https://searchfox.org/mozilla-central/source/memory/volatile...
rhn_mk1 7y ago
Do you know which mechanism is used for this?
- magicalhippo 7y ago
  In addition to the MEM_RESET, MEM_RESET_UNDO mentioned, there's also OfferVirtualMemory[1], ReclaimVirtualMemory[2] to let the system potentially use the memory if it needs it. Unlike MEM_RESET, this allows you to prioritize the memory you offer, as well as provides some extra safeguards it seems. No idea if Firefox uses this or MEM_RESET. [1]: https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-offervirtualmemory https://docs.microsoft.com/en-us/windows/win32/api/memoryapi... [2]: https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-reclaimvirtualmemory https://docs.microsoft.com/en-us/windows/win32/api/memoryapi...
  - rhn_mk1 7y ago
    That seems to be something different; instead of getting the memory when some other application needs it, the OS receives the memory when the application (browser) doesn't need it any longer, so it can't be used for caching. This actually looks exactly like the malloc/free use case.
    - magicalhippo 7y ago
      How can't it be used for caching? Say you decode an image, then offer the memory backed by it to the OS. If you need to access the image data again you try to reclaim the memory, if it fails it's a cache miss.
      - rhn_mk1 7y ago
        You're right, the missing info I didn't notice was the return value.
globalproctd 7y ago
What I was seeing was the same in both cases. Firefox would be using the same amount of RAM either running by itself or alongside Ableton Live (and the performance of Ableton Live would suffer).

0xdead 7y ago

That's not true. Operating systems don't know if a process is a browser.

basch 7y ago
OS > Hey I need RAM Browser > OK we can purge these tabs first is how it should work. If the first step doesnt happen, its not clear if the second one should execute by itself. Id be much more concerned about background CPU usage. Eats battery, and actually slows the computer down. That all said, its pretty clear that browsers are leaking RAM sometimes. I remember bed bath and beyond using 9 or 12GB of ram the other month.
- blattimwind 7y ago
  > is how it should work. That's not how it works. Browser might monitor free system memory actively and release memory if the system runs low, but at least Firefox doesn't do that. There is no mechanism (in Windows, Linux) that allows the OS to notify applications of memory pressure or that allows the application to tell the OS that some allocations can be thrown out if need be. (Windows has a mechanism - MEM_RESET - that only works with paged out allocations) Edit: Windows 10 (8?) actually expands on MEM_RESET with Offer/ReclaimVirtualMemory, but I kinda doubt browsers use this, because these have similar semantics to MEM_RESET (except they seem to work without paging, which is good). For an application that wants to cache not-well-defined amounts of data itself these might be very useful though, because you can make the data structure simple enough that it works with these.
  - zozbot234 7y ago
    > ...that allows the OS to notify applications of memory pressure or that allows the application to tell the OS that some allocations can be thrown out if need be. There are mechanisms for both of these in up-to-date Linux - pressure stall information and volatile ranges, respectively.
    - blattimwind 7y ago
      MADV_FREE didn't work properly until fairly recently, though, because it behaved like Windows' MEM_RESET. AIUI MADV_FREE is not intended for ranges that you actually want to work with (i.e. you want to keep the data, not just the virtual addresses), but rather as an optimization for allocators.
  - mauricioc 7y ago
    https://dblohm7.ca/blog/2015/07/28/interesting-win32-apis/ https://dblohm7.ca/blog/2015/07/28/interesting-win32-apis/ notes that Windows has the required APIs, and that Firefox already supports handling memory pressure notifications on non-Windows platforms. Not sure how up-to-date this information is.
  - kevingadd 7y ago
    Firefox does have a mechanism for reducing memory pressure, there's a button for it in about:memory. I can't guarantee that this flow actually happens upon low memory states on Windows etc but it wouldn't be hard to at least hack it together. Monitoring available memory (say, on a timer) is not a difficult thing to do on Windows - it would just be difficult to make it completely robust.
    - blattimwind 7y ago
      Well it definitely doesn't work on Linux, at least not automatically.
      - kevingadd 7y ago
        It's important to specify your test scenario here and the behavior you expect, as it's a known issue in general that Linux behaves poorly under OOM conditions. It's not necessarily Firefox specific. If all of my memory is "in use" but half of it is cached file pages, should Firefox compact its heap and evict cached images? What if all of my memory is spoken for but a bunch of it isn't actually committed, due to overcommit?
- glandium 7y ago
  Another strategy is: Browser> So this memory here, I don't really care if you swipe it in the future. (later) OS> Yum yum (later) Browser> Oh, that buffer is empty, let me fill it again. Firefox uses it, IIRC, for decoded image buffers. https://searchfox.org/mozilla-central/source/memory/volatile/VolatileBuffer.h https://searchfox.org/mozilla-central/source/memory/volatile...

rhn_mk1 7y ago

Do you know which mechanism is used for this?

magicalhippo 7y ago
In addition to the MEM_RESET, MEM_RESET_UNDO mentioned, there's also OfferVirtualMemory[1], ReclaimVirtualMemory[2] to let the system potentially use the memory if it needs it. Unlike MEM_RESET, this allows you to prioritize the memory you offer, as well as provides some extra safeguards it seems. No idea if Firefox uses this or MEM_RESET. [1]: https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-offervirtualmemory https://docs.microsoft.com/en-us/windows/win32/api/memoryapi... [2]: https://docs.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-reclaimvirtualmemory https://docs.microsoft.com/en-us/windows/win32/api/memoryapi...
- rhn_mk1 7y ago
  That seems to be something different; instead of getting the memory when some other application needs it, the OS receives the memory when the application (browser) doesn't need it any longer, so it can't be used for caching. This actually looks exactly like the malloc/free use case.
  - magicalhippo 7y ago
    How can't it be used for caching? Say you decode an image, then offer the memory backed by it to the OS. If you need to access the image data again you try to reclaim the memory, if it fails it's a cache miss.
    - rhn_mk1 7y ago
      You're right, the missing info I didn't notice was the return value.

globalproctd 7y ago

What I was seeing was the same in both cases. Firefox would be using the same amount of RAM either running by itself or alongside Ableton Live (and the performance of Ableton Live would suffer).