4 ms·

Wait... that means a misbehaving program can cause out of memory errors easily by filling up /tmp? That's a very bad default.

by api 1y ago

paulv 1y ago
The default configuration for tmpfs is to "only" use 50% of physical ram, which still isn't great, but it's something.
- foresto 1y ago
  To be clear, that 50% (or whatever you configure) is a limit, not a constant.
CamouflagedKiwi 1y ago
A misbehaving program can cause out of memory errors already by filling up memory. It wouldn't persist past that program's death but the effect is pretty catastrophic on other programs regardless.
- o11c 1y ago
  That actually is a pretty big difference. Assuming you're sane and have swap disabled (since there is no way to have a stable system with swap enabled), a program that tries to allocate all memory will quickly get OOM killed and the system will recover quickly. If /tmp/ fills up your RAM, the system will not recover automatically, and might not even be recoverable by hand without rebooting. That said, systemd-managed daemons using a private /tmp/ in RAM will correctly clear it when killed.
  - mxey 1y ago
    https://chrisdown.name/2018/01/02/in-defence-of-swap.html https://chrisdown.name/2018/01/02/in-defence-of-swap.html
    - o11c 1y ago
      An ivory tower answer. All those arguments would be useful if we somehow could avoid the fact that the system will use it as "emergency memory" and become unresponsive. The kernel's OOM killer is broken for this, and userland OOM daemons are unreliable. `vm.swappiness` is completely useless in the worst case, which is the only case that matters. With swap off, all the kernel needs to do is reserve a certain threshold for disk cache to avoid the thrashing problem. I don't know what the kernel actually does here (or what its tunables are), because systems with swap off have never caused problems for me the way systems with swap on inevitably do. The OOM killer works fine with swap off, because a system must always be resilient to unexpected process failure. And worst of all - the kernel requires swap (and its bugs) to be enabled for hibernation to work. It really wouldn't be hard to design a working swap system (just calculate how much to keep of different purposes of swap, and launch the OOM killer earlier), but apparently nobody in kernel-land understands the real-world problems enough to bother.
      - em-bee 1y ago
        the kernel requires swap (and its bugs) to be enabled for hibernation to work this one gets me irritated every time i think about it. i don't want to use swap, but i do want hibernation. why is there no way to disable swap without that? hmm, i suppose one could write a script that enables an inactive swap partition just before shutdown, and disables it again after boot.
        sigio 1y ago
        I never want to use hibernation, since then I have to re-enter my disk encryption passphrase at resume time, have to wait longer for both suspend and resume because it needs to sync upto 48GB to/from disk (and I don't want to waste 48GB of diskspace for swapspace/hibernation). Suspend to ram is fine, I can keep the system suspended for a couple of days without issues, but it only needs to survive a long weekend at most. Resume from RAM is about instant, and then just needs a screensaver unlock to get back to work.
        1y ago
  - tsimionescu 1y ago
    The sane thing is to have swap enabled. Having swap "disabled" forces your system to swap out executables to disk, since these are likely the only memory-mapped files you have. So, if your memory fills up, you get catastrophic thrashing of the instruction cache. If you're lucky, you really go over available memory, and the OOMKiller kills some random process. But if you're not, your system will keep chugging along at a snail's pace. Perhaps disabling overcommit as well as swap could be safer from this point of view. Unfortunately, you get other problems if you do so - as very little Linux software handles errors returned by malloc, since it's so uncommon to not have overcommit on a Linux system. I'd also note that swap isn't even that slow for SSDs, as long as you don't use it for code.
    - pmontra 1y ago
      I've been running with swap off since my first SSD in 2015 or 2016. 16 GB RAM, then 32. No problems at all. If I see RAM close to 30 GB I restart my browser and go back to 20 GB or less. Not every month.
      - progmetaldev 1y ago
        Are you running your own system for personal use, a service available to the public, or both? Do you normally see your system used consistently, or does it get used differently (and in random ways)? Since you state you're running a browser, I assume you mean for personal use. Unfortunately, when you run a service open to the public, you can find all kinds of odd traffic even for normal low-memory services. Sometimes you'll get hit with an aggressive bot looking for an exploit, and a lot of those bots don't care if they get blocked, because they are built to absolutely crush a system with exploits or login attempts where they are only blocked by the system crashing. I'd say that most bots are this aggressive, because the old school "script kiddies", or now it's just AI-enabled aggressors, just run code without understanding things. It's easier than ever to run an attack against a range of IP addresses looking for vulnerabilities, that can be chained into a LLM to generate code that can be run easily.
        pmontra 1y ago
  - nonameiguess 1y ago
    User paulv already posted this 3 hours ago in a comment currently lower than this one, but tmpfs by default can't use all of your RAM. /tmp can get filled up and be unavailable for anything else to write to, but you'll still have memory. It won't crash the entire system.
  - NekkoDroid 1y ago
    > If /tmp/ fills up your RAM tmpfs by default only uses up to half your available RAM unless specified otherwise. So this isn't really a consideration unless you configure it to be a consideration you need to take into account. (Systemd also really recently (v258) added quotas to tmpfs and IIRC its set by default to 80% of the tmpfs, so it is even less of a problem)
    - tremon 1y ago
      $ grep tmpfs /proc/mounts udev /dev devtmpfs [..] tmpfs /run tmpfs [..] tmpfs /run/lock tmpfs [..] tmpfs /run/shm tmpfs [..] tmpfs /tmp tmpfs [..] cgroup_root /sys/fs/cgroup tmpfs [..] If each of those can take up 50% of ram, this is still a big problem. I don't know what defaults Debian uses nowadays, because I have TMPFS_SIZE=1% in /etc/default/tmpfs so my system is explicitly non-default.
      - NekkoDroid 1y ago
        Sure, but counterpoint: if a process is already writing that much in multiple of those directories, who knows what its writing in other directories that aren't backed by RAM.
  - bigstrat2003 1y ago
    > Assuming you're sane and have swap disabled (since there is no way to have a stable system with swap enabled) What the heck are you talking about? Swap is enabled on every Linux system I manage (servers, desktop etc) and it's perfectly stable.

Wait... that means a misbehaving program can cause out of memory errors easily by filling up /tmp? That's a very bad default.

by api 1y ago

Wait... that means a misbehaving program can cause out of memory errors easily by filling up /tmp? That's a very bad default.

paulv 1y ago

The default configuration for tmpfs is to "only" use 50% of physical ram, which still isn't great, but it's something.

foresto 1y ago
To be clear, that 50% (or whatever you configure) is a limit, not a constant.

CamouflagedKiwi 1y ago

A misbehaving program can cause out of memory errors already by filling up memory. It wouldn't persist past that program's death but the effect is pretty catastrophic on other programs regardless.

o11c 1y ago
That actually is a pretty big difference. Assuming you're sane and have swap disabled (since there is no way to have a stable system with swap enabled), a program that tries to allocate all memory will quickly get OOM killed and the system will recover quickly. If /tmp/ fills up your RAM, the system will not recover automatically, and might not even be recoverable by hand without rebooting. That said, systemd-managed daemons using a private /tmp/ in RAM will correctly clear it when killed.
- mxey 1y ago
  https://chrisdown.name/2018/01/02/in-defence-of-swap.html https://chrisdown.name/2018/01/02/in-defence-of-swap.html
  - o11c 1y ago
    An ivory tower answer. All those arguments would be useful if we somehow could avoid the fact that the system will use it as "emergency memory" and become unresponsive. The kernel's OOM killer is broken for this, and userland OOM daemons are unreliable. `vm.swappiness` is completely useless in the worst case, which is the only case that matters. With swap off, all the kernel needs to do is reserve a certain threshold for disk cache to avoid the thrashing problem. I don't know what the kernel actually does here (or what its tunables are), because systems with swap off have never caused problems for me the way systems with swap on inevitably do. The OOM killer works fine with swap off, because a system must always be resilient to unexpected process failure. And worst of all - the kernel requires swap (and its bugs) to be enabled for hibernation to work. It really wouldn't be hard to design a working swap system (just calculate how much to keep of different purposes of swap, and launch the OOM killer earlier), but apparently nobody in kernel-land understands the real-world problems enough to bother.
    - em-bee 1y ago
      the kernel requires swap (and its bugs) to be enabled for hibernation to work this one gets me irritated every time i think about it. i don't want to use swap, but i do want hibernation. why is there no way to disable swap without that? hmm, i suppose one could write a script that enables an inactive swap partition just before shutdown, and disables it again after boot.
      - sigio 1y ago
        I never want to use hibernation, since then I have to re-enter my disk encryption passphrase at resume time, have to wait longer for both suspend and resume because it needs to sync upto 48GB to/from disk (and I don't want to waste 48GB of diskspace for swapspace/hibernation). Suspend to ram is fine, I can keep the system suspended for a couple of days without issues, but it only needs to survive a long weekend at most. Resume from RAM is about instant, and then just needs a screensaver unlock to get back to work.
        1y ago
- tsimionescu 1y ago
  The sane thing is to have swap enabled. Having swap "disabled" forces your system to swap out executables to disk, since these are likely the only memory-mapped files you have. So, if your memory fills up, you get catastrophic thrashing of the instruction cache. If you're lucky, you really go over available memory, and the OOMKiller kills some random process. But if you're not, your system will keep chugging along at a snail's pace. Perhaps disabling overcommit as well as swap could be safer from this point of view. Unfortunately, you get other problems if you do so - as very little Linux software handles errors returned by malloc, since it's so uncommon to not have overcommit on a Linux system. I'd also note that swap isn't even that slow for SSDs, as long as you don't use it for code.
  - pmontra 1y ago
    I've been running with swap off since my first SSD in 2015 or 2016. 16 GB RAM, then 32. No problems at all. If I see RAM close to 30 GB I restart my browser and go back to 20 GB or less. Not every month.
    - progmetaldev 1y ago
      Are you running your own system for personal use, a service available to the public, or both? Do you normally see your system used consistently, or does it get used differently (and in random ways)? Since you state you're running a browser, I assume you mean for personal use. Unfortunately, when you run a service open to the public, you can find all kinds of odd traffic even for normal low-memory services. Sometimes you'll get hit with an aggressive bot looking for an exploit, and a lot of those bots don't care if they get blocked, because they are built to absolutely crush a system with exploits or login attempts where they are only blocked by the system crashing. I'd say that most bots are this aggressive, because the old school "script kiddies", or now it's just AI-enabled aggressors, just run code without understanding things. It's easier than ever to run an attack against a range of IP addresses looking for vulnerabilities, that can be chained into a LLM to generate code that can be run easily.
      - pmontra 1y ago
- nonameiguess 1y ago
  User paulv already posted this 3 hours ago in a comment currently lower than this one, but tmpfs by default can't use all of your RAM. /tmp can get filled up and be unavailable for anything else to write to, but you'll still have memory. It won't crash the entire system.
- NekkoDroid 1y ago
  > If /tmp/ fills up your RAM tmpfs by default only uses up to half your available RAM unless specified otherwise. So this isn't really a consideration unless you configure it to be a consideration you need to take into account. (Systemd also really recently (v258) added quotas to tmpfs and IIRC its set by default to 80% of the tmpfs, so it is even less of a problem)
  - tremon 1y ago
    $ grep tmpfs /proc/mounts udev /dev devtmpfs [..] tmpfs /run tmpfs [..] tmpfs /run/lock tmpfs [..] tmpfs /run/shm tmpfs [..] tmpfs /tmp tmpfs [..] cgroup_root /sys/fs/cgroup tmpfs [..] If each of those can take up 50% of ram, this is still a big problem. I don't know what defaults Debian uses nowadays, because I have TMPFS_SIZE=1% in /etc/default/tmpfs so my system is explicitly non-default.
    - NekkoDroid 1y ago
      Sure, but counterpoint: if a process is already writing that much in multiple of those directories, who knows what its writing in other directories that aren't backed by RAM.
- bigstrat2003 1y ago
  > Assuming you're sane and have swap disabled (since there is no way to have a stable system with swap enabled) What the heck are you talking about? Swap is enabled on every Linux system I manage (servers, desktop etc) and it's perfectly stable.