4 ms·

> running without overcommit is a recipe for disaster (but yes, it probably is, even with 40GB of memory. I bet many things are designed on Linux with overcomm

by jraph 3y ago

> running without overcommit is a recipe for disaster (but yes, it probably is, even with 40GB of memory. I bet many things are designed on Linux with overcommit as an assumption)

LoganDark 3y ago
Well, that was just the wrong answer, since their software is supposed to be releasing memory back to the OS properly. Overcommit is just a coping mechanism, you should be addressing the root cause. I have since enabled the page file for other reasons - LLMs demand up to 50GB of memory sometimes, and my new desktop only has 16. The change of machine is why I should probably try Firefox again to see if it behaves.
- ColonelPhantom 3y ago
  Overcommit != Swap. Overcommit is allocating virtual memory without any backing. Swap is allocating physical memory backed by disk. Overcommit is useful in some cases, for example to preallocate a large heap without immediately making it all resident. Or to allocate 'guard' pages to fight buffer overflows. On Linux, overcommit is commonly assumed and as such disabling it tends to break some programs, as it's not out of the ordinary for something to allocate 100s of GBs of virtual memory.
  - LoganDark 3y ago
    Read up on Windows. Windows does not do overcommit whatsoever, unless swap is enabled, in which case it only allows overcommit up to the size of the swap file. Overcommit cannot be enabled without a swap file, whatsoever. This differs from Linux that can tend to have overcommit enabled without swap.
    - fomine3 3y ago
      You're correct, that's why having pagefile is important for Windows even if there's enough RAM. Why don't you enable it?
      - LoganDark 3y ago
        It takes up disk space that I thought I could save. I have it enabled now, as a side effect of working with LLMs on my new computer that only has 16GB of memory (working on fixing that). I still wouldn't be comfy letting Firefox fill it up, because no matter how large the page file is, your system will always crash when Firefox fills the entire thing. I do not know if my new computer will have this problem, though.
    - ColonelPhantom 3y ago
      Yes, Windows doesn't have overcommit. (Also not with swap, since overcommit is unbacked virtual memory, which Windows still doesn't allow. The only thing it allows is disk-backed virtual memory). But as a user, I don't care (except that I don't have to worry about an OOM killer because an allocation will just fail). The only real difference is that application developers need to be careful with allocating memory without using it, unlike on Unix-likes. Because software on Linux runs on the assumption of overcommit, you shouldn't disable it, even though the lack of overcommit on Windows is not problematic.
- mort96 3y ago
  The reasons browsers rely on overcommit has nothing to do with them failing to release memory back to the OS properly.
  - LoganDark 3y ago
    If the memory is never released back to the OS after it is done being used (or when it is not going to be used), then the browser has failed to release it back to the OS.
    - mort96 3y ago
      The memory is released back to the OS after it is done being used.
      - LoganDark 3y ago
        If that were the case my OS would not crash due to tens of gigabytes of allocated-but-not-used memory from Firefox
ta1243 3y ago
My laptop has a mere 32G of ram and 6G swap (currently unused but I did reboot last week) The advice used to be 1.5*physical ram for swap back in the days when I had 4MB of ram and 170MB HDD - well under 5% of disk space Doing that today would be 20% of my disk space which seemed excessive. Perhaps I could create a 24G swap file as a ram drive, giving me 8G of ram and 24G of swap. Why would that be better than just 32G of ram and no swap? Is it better than 40G of ram and no swap?
- ddalex 3y ago
  I stopped worrying about swap when I learned how to use zram.
- jraph 3y ago
  > 1.5*physical Yep. This was a rule of thumb which I don't think still holds with current RAM capacities. I don't have swap on my 32G laptop, not sure it's right I guess I could use a 2G-4G swap partition when running a few XWiki instances and a Java IDE as well as a browser with several dozens of tabs instead of having the IDE frequently OOM-killed.

by jraph 3y ago

> running without overcommit is a recipe for disaster (but yes, it probably is, even with 40GB of memory. I bet many things are designed on Linux with overcommit as an assumption)

LoganDark 3y ago

Well, that was just the wrong answer, since their software is supposed to be releasing memory back to the OS properly. Overcommit is just a coping mechanism, you should be addressing the root cause. I have since enabled the page file for other reasons - LLMs demand up to 50GB of memory sometimes, and my new desktop only has 16. The change of machine is why I should probably try Firefox again to see if it behaves.

ColonelPhantom 3y ago
Overcommit != Swap. Overcommit is allocating virtual memory without any backing. Swap is allocating physical memory backed by disk. Overcommit is useful in some cases, for example to preallocate a large heap without immediately making it all resident. Or to allocate 'guard' pages to fight buffer overflows. On Linux, overcommit is commonly assumed and as such disabling it tends to break some programs, as it's not out of the ordinary for something to allocate 100s of GBs of virtual memory.
- LoganDark 3y ago
  Read up on Windows. Windows does not do overcommit whatsoever, unless swap is enabled, in which case it only allows overcommit up to the size of the swap file. Overcommit cannot be enabled without a swap file, whatsoever. This differs from Linux that can tend to have overcommit enabled without swap.
  - fomine3 3y ago
    You're correct, that's why having pagefile is important for Windows even if there's enough RAM. Why don't you enable it?
    - LoganDark 3y ago
      It takes up disk space that I thought I could save. I have it enabled now, as a side effect of working with LLMs on my new computer that only has 16GB of memory (working on fixing that). I still wouldn't be comfy letting Firefox fill it up, because no matter how large the page file is, your system will always crash when Firefox fills the entire thing. I do not know if my new computer will have this problem, though.
  - ColonelPhantom 3y ago
    Yes, Windows doesn't have overcommit. (Also not with swap, since overcommit is unbacked virtual memory, which Windows still doesn't allow. The only thing it allows is disk-backed virtual memory). But as a user, I don't care (except that I don't have to worry about an OOM killer because an allocation will just fail). The only real difference is that application developers need to be careful with allocating memory without using it, unlike on Unix-likes. Because software on Linux runs on the assumption of overcommit, you shouldn't disable it, even though the lack of overcommit on Windows is not problematic.
mort96 3y ago
The reasons browsers rely on overcommit has nothing to do with them failing to release memory back to the OS properly.
- LoganDark 3y ago
  If the memory is never released back to the OS after it is done being used (or when it is not going to be used), then the browser has failed to release it back to the OS.
  - mort96 3y ago
    The memory is released back to the OS after it is done being used.
    - LoganDark 3y ago
      If that were the case my OS would not crash due to tens of gigabytes of allocated-but-not-used memory from Firefox

ta1243 3y ago

My laptop has a mere 32G of ram and 6G swap (currently unused but I did reboot last week) The advice used to be 1.5*physical ram for swap back in the days when I had 4MB of ram and 170MB HDD - well under 5% of disk space Doing that today would be 20% of my disk space which seemed excessive. Perhaps I could create a 24G swap file as a ram drive, giving me 8G of ram and 24G of swap. Why would that be better than just 32G of ram and no swap? Is it better than 40G of ram and no swap?

ddalex 3y ago
I stopped worrying about swap when I learned how to use zram.
jraph 3y ago
> 1.5*physical Yep. This was a rule of thumb which I don't think still holds with current RAM capacities. I don't have swap on my 32G laptop, not sure it's right I guess I could use a 2G-4G swap partition when running a few XWiki instances and a Java IDE as well as a browser with several dozens of tabs instead of having the IDE frequently OOM-killed.