45 ms·

I'd love to switch to ZFS, but the RAM requirements are absurd. I don't have a separate storage server, and I'm not really to sacrifice 10GB of RAM (1GB/TB of s

I'd love to switch to ZFS, but the RAM requirements are absurd. I don't have a separate storage server, and I'm not really to sacrifice 10GB of RAM (1GB/TB of storage if I'm to believe what I find through Google) on my home desktop just for it when the vast majority of my data could probably handle a rotted bit or two. I did briefly try ZFS on my laptop a year or so ago, and it ate up half of my RAM permanently. Since it was already fairly limited, that wasn't a sacrifice I wasn't willing to make either when I have plenty of backups anyway.

vetinari 6y ago
That RAM is needed only when you run deduplication (you have to store the checksums of blocks that you deduplicate somewhere). If you don't, the RAM requirements are similar to other filesystems. On your home desktop, you don't have to run dedup. You will get still the bitrot protection.
- Mekantis 6y ago
  Why wouldn't I want deduplication though? "You can use ZFS just fine, just turn off one of its most useful features." Really? And I'm being downvoted for it. Thanks, guys. You realize my home desktop is doubling as my storage, right? Which goes back to RAM requirements being an issue. I hate the cargoculting on this fucking site.
  - vetinari 6y ago
    > Why wouldn't I want deduplication though? On a desktop? Chances are, that you don't have many users saving the same or slightly modified version of a file on the same storage. For a single person, it doesn't make much sense. > "You can use ZFS just fine, just turn off one of its most useful features." ZFS has many useful features. They come with a price though, because there's no free computation (see also laws of thermodynamics). It is then a matter of deciding, which features you want or need, and are willing to pay the price for. You obviously are not willing the pay the price for dedup (lvmvdo asks for similar price, so it is not ZFS-specific), so why are you complaining that you cannot use it? ZFS still has many more useful features. You also have another option: add RAM to your desktop. It is cheap. Then you will be able to use that one feature. > I hate the cargoculting on this fucking site. Sigh. I'm actually btrfs fan, all my data are on a btrfs volume (at work, we do use ZFS though, so I do have the experience). But that doesn't mean I won't point out something that the other club does well.
  - fomine3 6y ago
    No one agree that dedup is most useful feature on ZFS. It is generally considered as useless feature unless you have special datasets.
ahofmann 6y ago
That zfs needs so much RAM is a myth that can be easily refuted. Start a Linux VM with hardly any RAM and let zfs run on it. Works great. There are also sources for this since years: https://www.reddit.com/r/DataHoarder/comments/3s7vrd/so_you_think_zfs_needs_a_ton_of_ram_for_a_simple/ https://www.reddit.com/r/DataHoarder/comments/3s7vrd/so_you_...
- Mekantis 6y ago
  Look at the comments. He has barely any files on that server. "Yeah, you can use ZFS with barely any RAM." (if you don't have any files) Doesn't do me much good with my 10TB's of data.
magicalhippo 6y ago
> I did briefly try ZFS on my laptop a year or so ago, and it ate up half of my RAM permanently. ZFS will use all available RAM for caching, unless you tell it otherwise (zfs_arc_max[1] or similar[2]), but it should release it when the system requires it[3]. [1]: https://docs.oracle.com/cd/E19253-01/817-0404/gjhec/index.html https://docs.oracle.com/cd/E19253-01/817-0404/gjhec/index.ht... [2]: https://www.freebsd.org/doc/handbook/zfs-advanced.html https://www.freebsd.org/doc/handbook/zfs-advanced.html [3]: https://superuser.com/a/1137417 https://superuser.com/a/1137417
JeremyNT 6y ago
> I'd love to switch to ZFS, but the RAM requirements are absurd. I don't have a separate storage server, and I'm not really to sacrifice 10GB of RAM (1GB/TB of storage if I'm to believe what I find through Google) on my home desktop just for it when the vast majority of my data could probably handle a rotted bit or two. Please explain where this number comes from. I run ZFS on boxes with as little as 4GB of RAM, which are also doing all sorts of other things in addition to ZFS. As with all filesystems on Linux, more RAM means more cache, and if you have free RAM you will benefit from a dynamically resized filesystem cache. That RAM is however not required and the cache can be evicted under memory pressure.
- simcop2387 6y ago
  Its the recommemded setup if you want to use dedupe, aside from that it doesn't really meed more ram than any other fs.

vetinari 6y ago

That RAM is needed only when you run deduplication (you have to store the checksums of blocks that you deduplicate somewhere). If you don't, the RAM requirements are similar to other filesystems. On your home desktop, you don't have to run dedup. You will get still the bitrot protection.

Mekantis 6y ago
Why wouldn't I want deduplication though? "You can use ZFS just fine, just turn off one of its most useful features." Really? And I'm being downvoted for it. Thanks, guys. You realize my home desktop is doubling as my storage, right? Which goes back to RAM requirements being an issue. I hate the cargoculting on this fucking site.
- vetinari 6y ago
  > Why wouldn't I want deduplication though? On a desktop? Chances are, that you don't have many users saving the same or slightly modified version of a file on the same storage. For a single person, it doesn't make much sense. > "You can use ZFS just fine, just turn off one of its most useful features." ZFS has many useful features. They come with a price though, because there's no free computation (see also laws of thermodynamics). It is then a matter of deciding, which features you want or need, and are willing to pay the price for. You obviously are not willing the pay the price for dedup (lvmvdo asks for similar price, so it is not ZFS-specific), so why are you complaining that you cannot use it? ZFS still has many more useful features. You also have another option: add RAM to your desktop. It is cheap. Then you will be able to use that one feature. > I hate the cargoculting on this fucking site. Sigh. I'm actually btrfs fan, all my data are on a btrfs volume (at work, we do use ZFS though, so I do have the experience). But that doesn't mean I won't point out something that the other club does well.
- fomine3 6y ago
  No one agree that dedup is most useful feature on ZFS. It is generally considered as useless feature unless you have special datasets.

ahofmann 6y ago

That zfs needs so much RAM is a myth that can be easily refuted. Start a Linux VM with hardly any RAM and let zfs run on it. Works great. There are also sources for this since years: https://www.reddit.com/r/DataHoarder/comments/3s7vrd/so_you_think_zfs_needs_a_ton_of_ram_for_a_simple/ https://www.reddit.com/r/DataHoarder/comments/3s7vrd/so_you_...

Mekantis 6y ago
Look at the comments. He has barely any files on that server. "Yeah, you can use ZFS with barely any RAM." (if you don't have any files) Doesn't do me much good with my 10TB's of data.

magicalhippo 6y ago

> I did briefly try ZFS on my laptop a year or so ago, and it ate up half of my RAM permanently. ZFS will use all available RAM for caching, unless you tell it otherwise (zfs_arc_max[1] or similar[2]), but it should release it when the system requires it[3]. [1]: https://docs.oracle.com/cd/E19253-01/817-0404/gjhec/index.html https://docs.oracle.com/cd/E19253-01/817-0404/gjhec/index.ht... [2]: https://www.freebsd.org/doc/handbook/zfs-advanced.html https://www.freebsd.org/doc/handbook/zfs-advanced.html [3]: https://superuser.com/a/1137417 https://superuser.com/a/1137417

JeremyNT 6y ago

> I'd love to switch to ZFS, but the RAM requirements are absurd. I don't have a separate storage server, and I'm not really to sacrifice 10GB of RAM (1GB/TB of storage if I'm to believe what I find through Google) on my home desktop just for it when the vast majority of my data could probably handle a rotted bit or two. Please explain where this number comes from. I run ZFS on boxes with as little as 4GB of RAM, which are also doing all sorts of other things in addition to ZFS. As with all filesystems on Linux, more RAM means more cache, and if you have free RAM you will benefit from a dynamically resized filesystem cache. That RAM is however not required and the cache can be evicted under memory pressure.

simcop2387 6y ago
Its the recommemded setup if you want to use dedupe, aside from that it doesn't really meed more ram than any other fs.