4 ms·

Why disable swap? And...only 8gb of ram (am I misreading that?) with zfs? It's been years since I've needed a bunch of storage but I remember zfs being much hap

Why disable swap? And...only 8gb of ram (am I misreading that?) with zfs? It's been years since I've needed a bunch of storage but I remember zfs being much happier with gobs of ram.

e12e 3y ago
IIRCC it's not recommend to have swap on a zvol. So you would need a non-zfs managed partition/device for swap. Not sure about current status? https://github.com/openzfs/zfs/issues/7734 https://github.com/openzfs/zfs/issues/7734 https://github.com/openzfs/zfs/issues/342 https://github.com/openzfs/zfs/issues/342 https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSForSwapMyViews https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSForSwapM...
- temp0826 3y ago
  Swap on a zvol I understand, disabling it entirely seems questionable, no?
  - psanford 3y ago
    This is a dedicated NAS. What workload do you expect to be paging to disk? Your primary workload is already reading and writing bits off disk.
    - temp0826 3y ago
      Because we live in the real world and no system is truly in a bubble if it's connected to a network. But mostly because seeing oom-killer messages in dmesg makes me sad :).
      - Dylan16807 3y ago
        If something gets leaky enough then it's still going to hurt performance for a while and die. Swap delays that but extra RAM delays it too. If you take a use case where 2GB of memory is fine, and give it 8GB, then you already solved the problem swap would solve. You can always add more but you're past the point of diminishing returns.
  - Dylan16807 3y ago
    Since there's already a non-ZFS system disk, it's a little weird to not put a swap partition on it. But I wouldn't say it's questionable. On a NAS I doubt you'd ever use even one full gigabyte of swap space. Keeping it simple is fine.
- usefulcat 3y ago
  He already has a separate NVMe drive for the OS, he could put the swap on that.
  - e12e 3y ago
    Only if he's not running rootfs on zfs? Ed: I see he doesn't - but could (and optionally with a slice for swap partition): https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubuntu/Ubuntu%2022.04%20Root%20on%20ZFS.html#overview https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubu...
squarefoot 3y ago
I'm using XigmaNAS with two zfs pools and only 4G RAM since like 8 years, also on a seemingly underpowered machine (Atom D410). Granted that it doesn't fly, but for home use is more than enough say for serving on the home LAN multiple HD movies at the same time, provided one doesn't load it with other more heavy services. Mine is currently only serving files through NFS and SMB and bittorrent; all else is turned off. https://ibb.co/BNMWFXm https://ibb.co/BNMWFXm BTW, currently working around a bigger one that will run on a faster mini PC and a 8 bay USB3.1 enclosure. I'm a little wary about USB connectors in this context, and I'll likely have to secure cables firmly to avoid wearing and accidental pulls, but so far results on the bench are promising.
Modified3019 3y ago
ZFS doesn’t need much ram unless you have specific needs. People have run basic zfs storage machines on 4GB of memory total and it works fine. Even 2GB has been done before, but that’s a bit too low for me to suggest. Broadly, 16GB of total system is personal general recommended baseline, because honestly it’s simply too cheap not to have it. ZFS needs some amount of ram in order to even load the pool, this only becomes a practical concern when your pool gets into the 100’s of TiB. Deduplication, which should generally not be used, used to be awful with ram consumption, but these days isn’t nearly as bad and can be surprisingly viable. Dedupe can still cause unexpected performance issues unless your skillset is into digging into system analysis and tuning. You need some amount of RAM buffer for the write TXGs to coalesce efficiently. Generally not a concern. Finally there’s ARC, which is where all the nice things that improve your experience happen. The more the better, but just like system ram, once you have enough for your usage profile, you stop noticing much benefit beyond that. For dumb file storage, not that much is needed. Ideally you want enough to keep all the metadata, plus whatever your actual repeated read access would be. Working on video editing and VMs would require more RAM for a more optimal experience.
- temp0826 3y ago
  Like I mentioned it's been a long time since I touched this stuff. I do think I set up my (underpowered) system aggressively with ARC now that you mention it. Probably was experimenting with dedup as well. I'm sure some of those things have had improvements over the years.

Why disable swap? And...only 8gb of ram (am I misreading that?) with zfs? It's been years since I've needed a bunch of storage but I remember zfs being much happier with gobs of ram.

e12e 3y ago

IIRCC it's not recommend to have swap on a zvol. So you would need a non-zfs managed partition/device for swap. Not sure about current status? https://github.com/openzfs/zfs/issues/7734 https://github.com/openzfs/zfs/issues/7734 https://github.com/openzfs/zfs/issues/342 https://github.com/openzfs/zfs/issues/342 https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSForSwapMyViews https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSForSwapM...

temp0826 3y ago
Swap on a zvol I understand, disabling it entirely seems questionable, no?
- psanford 3y ago
  This is a dedicated NAS. What workload do you expect to be paging to disk? Your primary workload is already reading and writing bits off disk.
  - temp0826 3y ago
    Because we live in the real world and no system is truly in a bubble if it's connected to a network. But mostly because seeing oom-killer messages in dmesg makes me sad :).
    - Dylan16807 3y ago
      If something gets leaky enough then it's still going to hurt performance for a while and die. Swap delays that but extra RAM delays it too. If you take a use case where 2GB of memory is fine, and give it 8GB, then you already solved the problem swap would solve. You can always add more but you're past the point of diminishing returns.
- Dylan16807 3y ago
  Since there's already a non-ZFS system disk, it's a little weird to not put a swap partition on it. But I wouldn't say it's questionable. On a NAS I doubt you'd ever use even one full gigabyte of swap space. Keeping it simple is fine.
usefulcat 3y ago
He already has a separate NVMe drive for the OS, he could put the swap on that.
- e12e 3y ago
  Only if he's not running rootfs on zfs? Ed: I see he doesn't - but could (and optionally with a slice for swap partition): https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubuntu/Ubuntu%2022.04%20Root%20on%20ZFS.html#overview https://openzfs.github.io/openzfs-docs/Getting%20Started/Ubu...

squarefoot 3y ago

I'm using XigmaNAS with two zfs pools and only 4G RAM since like 8 years, also on a seemingly underpowered machine (Atom D410). Granted that it doesn't fly, but for home use is more than enough say for serving on the home LAN multiple HD movies at the same time, provided one doesn't load it with other more heavy services. Mine is currently only serving files through NFS and SMB and bittorrent; all else is turned off. https://ibb.co/BNMWFXm https://ibb.co/BNMWFXm BTW, currently working around a bigger one that will run on a faster mini PC and a 8 bay USB3.1 enclosure. I'm a little wary about USB connectors in this context, and I'll likely have to secure cables firmly to avoid wearing and accidental pulls, but so far results on the bench are promising.

Modified3019 3y ago

ZFS doesn’t need much ram unless you have specific needs. People have run basic zfs storage machines on 4GB of memory total and it works fine. Even 2GB has been done before, but that’s a bit too low for me to suggest. Broadly, 16GB of total system is personal general recommended baseline, because honestly it’s simply too cheap not to have it. ZFS needs some amount of ram in order to even load the pool, this only becomes a practical concern when your pool gets into the 100’s of TiB. Deduplication, which should generally not be used, used to be awful with ram consumption, but these days isn’t nearly as bad and can be surprisingly viable. Dedupe can still cause unexpected performance issues unless your skillset is into digging into system analysis and tuning. You need some amount of RAM buffer for the write TXGs to coalesce efficiently. Generally not a concern. Finally there’s ARC, which is where all the nice things that improve your experience happen. The more the better, but just like system ram, once you have enough for your usage profile, you stop noticing much benefit beyond that. For dumb file storage, not that much is needed. Ideally you want enough to keep all the metadata, plus whatever your actual repeated read access would be. Working on video editing and VMs would require more RAM for a more optimal experience.

temp0826 3y ago
Like I mentioned it's been a long time since I touched this stuff. I do think I set up my (underpowered) system aggressively with ARC now that you mention it. Probably was experimenting with dedup as well. I'm sure some of those things have had improvements over the years.