4 ms·

The third mitigating feature the article forgot to mention is that tmpfs can get paged out to the swap partition. If you drop a large file there and forget it,

The third mitigating feature the article forgot to mention is that tmpfs can get paged out to the swap partition. If you drop a large file there and forget it, it will all end up in the swap partition if applications are demanding more memory.

buckle8017 1y ago
Which is a great reason to have a big swap file now.
- gnyman 1y ago
  Note though that if you don't have swap now, and enable it, you introduce the risk of thrashing [1] If you have swap already it doesn't matter, but I've encountered enough thrashing that I now disable swap on almost all servers I work with. It's rare but when it happens the server usually becomes completely unresponsive, so you have to hard reset it. I'd rather that the application trying to use too much memory is killed by the oom manager and I can ssh in and fix that. [1] https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/4/html/introduction_to_system_administration/s2-memory-concepts-swapping https://docs.redhat.com/en/documentation/red_hat_enterprise_...
  - baq 1y ago
    This is why I’m running with overcommit 2 and a different ratio per server purpose. …though I’m not sure why we have to think about this in 2025 at all.
    - worthless-trash 1y ago
      I'm assuming that you monitor the service closely for OOM then adjust with demand ?
      - baq 1y ago
        yeah pretty much, also configuring memory limits everywhere where apps allow it. some software also handles malloc failures relatively gracefully, which helps a whole lot (thank you postgres devs)
        worthless-trash 1y ago
        Ive spent the last day thinking about that, I really can't see any big negative side effects, the only issue that I'd have is being notified of OOM conditions, and that would just be a syslog regex match. Great plan.
  - k_bx 1y ago
    Disabling swap on servers is de-facto standard for serious deployments. The swap story needs a serious upgrade. I think /tmp in memory is a great idea, but I also think that particular /tmp needs a swap support (ideally with compression, ZSWAP), but not the main system.
    - ravetcofx 1y ago
      Swap always seemed more meant for desktop use. Servers you need to give the real memory expected of the application stack.
      - someothherguyy 1y ago
        plenty of footguns in that general advice, local in memory storage services with default config, etc
      - finaard 1y ago
        Pretty much all the guidelines about swap partitions out there reference old allocator behaviour from way over a decade ago - where you'd indeed typically run into weird issues without having a swap partition, even if you had enough RAM. Short (and inaccurate) summary was that it'd try to use some swap even if it didn't need it yet, which made sense in the world of enough memory being too expensive, and got fixed at the cost of making the allocator way more complicated when we started having enough memory in most cases. Nowadays typically you don't need swap unless you work on a product with some constraints, in which case you'd hand tune low memory performance anyway. Just don't buy anything with less than 32GB, and you should be good.
    - bmacho 1y ago
      So the ideal behaviour would be: - for most processes no SWAP - for tmpfs, use RAM until a quota - for tmpfs, start using a swapfile above that quota ChatGPT doesn't think it is achievable, though it thinks cgroup2 can achieve something similar.
    - throw0101c 1y ago
      > Disabling swap on servers is de-facto standard for serious deployments. I guess I have not been deploying seriously over the last couple of decades because the (hardware) systems that I deploy all had some swap, even if it was only a file.
  - deleted 1y ago
    [deleted]
  - mnw21cam 1y ago
    That's not true. Without swap, you already have the risk of thrashing. This is because Linux views all segments of code which your processes are running as clean and evictable from the cache, and therefore basically equivalent to swap, even when you have no swap. Under low-memory conditions, Linux will happily evict all clean pages, including the ones that the next process to be scheduled needs to execute from, causing thrashing. You can still get an unresponsive server under low memory conditions due to thrashing with no swap. Setting swappiness to zero doesn't fix this. Disabling swap doesn't fix this. Disabling overcommit does fix this, but that might have unacceptable disadvantages if some of the processes you are running allocate much more RAM than they use. Installing earlyoom to prevent real low memory conditions does fix this, and is probably the best solution.
- deleted 1y ago
  [deleted]
m463 1y ago
what swap partition? I meant this sort of jokingly. I think have a few linux systems that were never configured with swap partitions or swapfiles.
- edoceo 1y ago
  I'm with you. I don't swap. Processes die. OOM. Linux can recover and not lose data. Just unavailable for a moment.
  - Balinares 1y ago
    Swapping still occurs regardless. If there is no swap space the kernel swaps out code pages instead. So, running programs. The code pages then need to be loaded again from disk when the corresponding process is next scheduled and needs them. This is not very efficient and is why a bit of actual swap space is generally recommended.
    - adrian_b 1y ago
      Unlike swapping, freeing code pages does no writing to HDD/SSD, but it only needs to reload the pages when they are needed again in the future, therefore it is more efficient than swapping. I have stopped using swapping on all my Linux servers, desktops and laptops more than 20 years ago. At that time it was a great improvement and since then it has never caused any problems. However, I have been generous with the amount of RAM I install, for any computer having at least the NUC size there are many years since I have never used less than 32 GB, while for new computers I do not intend to use less than 64 GB. With recent enough Linux kernels, using tmpfs for /tmp is perfectly fine. Nevertheless, for decades using tmpfs for /tmp had been dangerous, because copying a file through /tmp would lose metadata, e.g. by truncating file timestamps and by stripping the extended file attributes. Copying files through /tmp was frequent between the users of multi-user computers where there was no other directory where all users had write access and the former behavior of Linux tmpfs was very surprising for them.
  - marginalia_nu 1y ago
    The Linux OOM killer is kinda sketchy to rely on. It likes to freeze up your system for long periods of time as it works out how to resolve the issue. Then it starts killing random PIDs to try to reclaim RAM like a system wide russian roulette. It's especially janky when you don't have swap. I've found adding a small swap file of ~500 MB makes it work so much better, even for systems with half a terabyte of RAM this helps reduce the freezing issues.
guappa 1y ago
Fedora did this long before debian. I remember doing wget of an .iso file on /tmp and my entire wayland session being killed by the OOM killer. I still think it's a terrible idea.
- nolist_policy 1y ago
  Use `/var/tmp` of you want a disk backed tmp.
  - 1718627440 1y ago
    I thought /var/tmp is for applications while /tmp is for the user.
    - Hendrikto 1y ago
      > /tmp/ > The place for small temporary files. This directory is usually mounted as a tmpfs instance, and should hence not be used for larger files. (Use /var/tmp/ for larger files.) This directory is usually flushed at boot-up. Also, files that are not accessed within a certain time may be automatically deleted. Source: https://uapi-group.org/specifications/specs/linux_file_system_hierarchy/ https://uapi-group.org/specifications/specs/linux_file_syste...
      - guappa 1y ago
        But that was written after the change was made :D
    - styanax 1y ago
      Trivia: CIS Guidelines (security tasks applied to a server to pass an enhanced security audit to be compliant with a standard, in a soundbite) has an item requiring /var/tmp to be a bind mount to /tmp (as well as setting specific security options on /tmp). A server attempting to pass CIS audits (very common in my work-related experience w/Enterprises) may well not have a unique /var/tmp.
    - 1y ago

buckle8017 1y ago

Which is a great reason to have a big swap file now.

gnyman 1y ago
Note though that if you don't have swap now, and enable it, you introduce the risk of thrashing [1] If you have swap already it doesn't matter, but I've encountered enough thrashing that I now disable swap on almost all servers I work with. It's rare but when it happens the server usually becomes completely unresponsive, so you have to hard reset it. I'd rather that the application trying to use too much memory is killed by the oom manager and I can ssh in and fix that. [1] https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/4/html/introduction_to_system_administration/s2-memory-concepts-swapping https://docs.redhat.com/en/documentation/red_hat_enterprise_...
- baq 1y ago
  This is why I’m running with overcommit 2 and a different ratio per server purpose. …though I’m not sure why we have to think about this in 2025 at all.
  - worthless-trash 1y ago
    I'm assuming that you monitor the service closely for OOM then adjust with demand ?
    - baq 1y ago
      yeah pretty much, also configuring memory limits everywhere where apps allow it. some software also handles malloc failures relatively gracefully, which helps a whole lot (thank you postgres devs)
      - worthless-trash 1y ago
        Ive spent the last day thinking about that, I really can't see any big negative side effects, the only issue that I'd have is being notified of OOM conditions, and that would just be a syslog regex match. Great plan.
- k_bx 1y ago
  Disabling swap on servers is de-facto standard for serious deployments. The swap story needs a serious upgrade. I think /tmp in memory is a great idea, but I also think that particular /tmp needs a swap support (ideally with compression, ZSWAP), but not the main system.
  - ravetcofx 1y ago
    Swap always seemed more meant for desktop use. Servers you need to give the real memory expected of the application stack.
    - someothherguyy 1y ago
      plenty of footguns in that general advice, local in memory storage services with default config, etc
    - finaard 1y ago
      Pretty much all the guidelines about swap partitions out there reference old allocator behaviour from way over a decade ago - where you'd indeed typically run into weird issues without having a swap partition, even if you had enough RAM. Short (and inaccurate) summary was that it'd try to use some swap even if it didn't need it yet, which made sense in the world of enough memory being too expensive, and got fixed at the cost of making the allocator way more complicated when we started having enough memory in most cases. Nowadays typically you don't need swap unless you work on a product with some constraints, in which case you'd hand tune low memory performance anyway. Just don't buy anything with less than 32GB, and you should be good.
  - bmacho 1y ago
    So the ideal behaviour would be: - for most processes no SWAP - for tmpfs, use RAM until a quota - for tmpfs, start using a swapfile above that quota ChatGPT doesn't think it is achievable, though it thinks cgroup2 can achieve something similar.
  - throw0101c 1y ago
    > Disabling swap on servers is de-facto standard for serious deployments. I guess I have not been deploying seriously over the last couple of decades because the (hardware) systems that I deploy all had some swap, even if it was only a file.
- deleted 1y ago
  [deleted]
- mnw21cam 1y ago
  That's not true. Without swap, you already have the risk of thrashing. This is because Linux views all segments of code which your processes are running as clean and evictable from the cache, and therefore basically equivalent to swap, even when you have no swap. Under low-memory conditions, Linux will happily evict all clean pages, including the ones that the next process to be scheduled needs to execute from, causing thrashing. You can still get an unresponsive server under low memory conditions due to thrashing with no swap. Setting swappiness to zero doesn't fix this. Disabling swap doesn't fix this. Disabling overcommit does fix this, but that might have unacceptable disadvantages if some of the processes you are running allocate much more RAM than they use. Installing earlyoom to prevent real low memory conditions does fix this, and is probably the best solution.
deleted 1y ago
[deleted]

m463 1y ago

what swap partition? I meant this sort of jokingly. I think have a few linux systems that were never configured with swap partitions or swapfiles.

edoceo 1y ago
I'm with you. I don't swap. Processes die. OOM. Linux can recover and not lose data. Just unavailable for a moment.
- Balinares 1y ago
  Swapping still occurs regardless. If there is no swap space the kernel swaps out code pages instead. So, running programs. The code pages then need to be loaded again from disk when the corresponding process is next scheduled and needs them. This is not very efficient and is why a bit of actual swap space is generally recommended.
  - adrian_b 1y ago
    Unlike swapping, freeing code pages does no writing to HDD/SSD, but it only needs to reload the pages when they are needed again in the future, therefore it is more efficient than swapping. I have stopped using swapping on all my Linux servers, desktops and laptops more than 20 years ago. At that time it was a great improvement and since then it has never caused any problems. However, I have been generous with the amount of RAM I install, for any computer having at least the NUC size there are many years since I have never used less than 32 GB, while for new computers I do not intend to use less than 64 GB. With recent enough Linux kernels, using tmpfs for /tmp is perfectly fine. Nevertheless, for decades using tmpfs for /tmp had been dangerous, because copying a file through /tmp would lose metadata, e.g. by truncating file timestamps and by stripping the extended file attributes. Copying files through /tmp was frequent between the users of multi-user computers where there was no other directory where all users had write access and the former behavior of Linux tmpfs was very surprising for them.
- marginalia_nu 1y ago
  The Linux OOM killer is kinda sketchy to rely on. It likes to freeze up your system for long periods of time as it works out how to resolve the issue. Then it starts killing random PIDs to try to reclaim RAM like a system wide russian roulette. It's especially janky when you don't have swap. I've found adding a small swap file of ~500 MB makes it work so much better, even for systems with half a terabyte of RAM this helps reduce the freezing issues.

guappa 1y ago

Fedora did this long before debian. I remember doing wget of an .iso file on /tmp and my entire wayland session being killed by the OOM killer. I still think it's a terrible idea.

nolist_policy 1y ago
Use `/var/tmp` of you want a disk backed tmp.
- 1718627440 1y ago
  I thought /var/tmp is for applications while /tmp is for the user.
  - Hendrikto 1y ago
    > /tmp/ > The place for small temporary files. This directory is usually mounted as a tmpfs instance, and should hence not be used for larger files. (Use /var/tmp/ for larger files.) This directory is usually flushed at boot-up. Also, files that are not accessed within a certain time may be automatically deleted. Source: https://uapi-group.org/specifications/specs/linux_file_system_hierarchy/ https://uapi-group.org/specifications/specs/linux_file_syste...
    - guappa 1y ago
      But that was written after the change was made :D
  - styanax 1y ago
    Trivia: CIS Guidelines (security tasks applied to a server to pass an enhanced security audit to be compliant with a standard, in a soundbite) has an item requiring /var/tmp to be a bind mount to /tmp (as well as setting specific security options on /tmp). A server attempting to pass CIS audits (very common in my work-related experience w/Enterprises) may well not have a unique /var/tmp.
  - 1y ago