4 ms·

Tarsnap looks pretty good for some really niche use cases, but it seems pretty crazy expensive if you don't need the ultra privacy guarantees. Is ZFS actually

Tarsnap looks pretty good for some really niche use cases, but it seems pretty crazy expensive if you don't need the ultra privacy guarantees. Is ZFS actually a possibility on 512MB of RAM? I always thought you needed actual server-class hardware way beyond a cheap consumer NAS for that, at least if you want the deduplication. It does look really nice, it would be super cool to be able to just deduplicate everything globally, even between backups of different machines.

ggm 2y ago
tarsnap is extremely expensive because it's pretty gold-class. If you can't afford it, then the majors (apple, google, microsoft) are probably your best bet because they do volume. Yes, you can run ZFS on machines with small memory, if the primary role is archive media. There is a huge misunderstanding about why ZFS gobbles memory which is to do with cacheing for fast access. When you run it as a clone-backup strategy, you're not doing the kind of workflows which demand a giant ARC or intent-log. But indeed you cannot run de-duplication on that, and there are people who say it's unwise anyway. I don't run it, I do run lz4 compression and noatime. With snapshots after the first copy, it's incrementals so the only blocks being written are against files with changes. There is some wastage, I agree but I think lz4 makes up for it. A counter case might be my partners windows backup (pre-11 which hides this stuff) which has a huge chain of Outlook files, mostly the same data, all slightly different. But with 12TB class disk, this becomes less of an issue. I moved to a zimablade with SATA card, from a pi4 with USB-SATA bridge. The ZFS migrated fine. I have a USB3 removable for my cold storage and write the ZFS incrementals to it periodically (monthly) I probably should have bought a 4 or 5 disk TrueNAS capable chassis, and I would aim for 1-2GB over 512 if you can. If its a single disk NAS, you're exposed to more weaknesses if you don't have something else. But I have to keep reminding myself a ZFS Raid is not a backup.
- eternityforest 2y ago
  I got the 512MB 223j mostly just because the refurb version seemed to be the cheapest well-reviewed, low power, and current model 2-bay NAS, and I assume the tock firmware is probably optimized to work well enough. I can think of a lot of cool extra possibilities with more memory though. I'm not seeing a ZFS option for Synology's stock firmware, and I was hoping to stick with their OS, but if it's vastly and overwhelmingly better I could see switching to something else. Are snapshots actually necessary? Do they provide something incrementals to the external drive don't? I guess they're lighter weight and you can do them more often?
  - ggm 2y ago
    If the FS implements snapshots "down low" you get COW efficient, and pretty much lock-free atomic "it's done" -if you run userspace "get me the difference" then it has all kinds of runtime costs and risks. That said, rsync is pretty good. Your J model can't get more memory. And you look to be right, its the OS it comes with and thats it. The NAS you have is the best NAS sometimes. It's all very well positing better ones, but they aren't the one you have: So make the most of it.
    - eternityforest 2y ago
      Yup, the best NAS is the one you can actually justify buying!

ggm 2y ago

tarsnap is extremely expensive because it's pretty gold-class. If you can't afford it, then the majors (apple, google, microsoft) are probably your best bet because they do volume. Yes, you can run ZFS on machines with small memory, if the primary role is archive media. There is a huge misunderstanding about why ZFS gobbles memory which is to do with cacheing for fast access. When you run it as a clone-backup strategy, you're not doing the kind of workflows which demand a giant ARC or intent-log. But indeed you cannot run de-duplication on that, and there are people who say it's unwise anyway. I don't run it, I do run lz4 compression and noatime. With snapshots after the first copy, it's incrementals so the only blocks being written are against files with changes. There is some wastage, I agree but I think lz4 makes up for it. A counter case might be my partners windows backup (pre-11 which hides this stuff) which has a huge chain of Outlook files, mostly the same data, all slightly different. But with 12TB class disk, this becomes less of an issue. I moved to a zimablade with SATA card, from a pi4 with USB-SATA bridge. The ZFS migrated fine. I have a USB3 removable for my cold storage and write the ZFS incrementals to it periodically (monthly) I probably should have bought a 4 or 5 disk TrueNAS capable chassis, and I would aim for 1-2GB over 512 if you can. If its a single disk NAS, you're exposed to more weaknesses if you don't have something else. But I have to keep reminding myself a ZFS Raid is not a backup.

eternityforest 2y ago
I got the 512MB 223j mostly just because the refurb version seemed to be the cheapest well-reviewed, low power, and current model 2-bay NAS, and I assume the tock firmware is probably optimized to work well enough. I can think of a lot of cool extra possibilities with more memory though. I'm not seeing a ZFS option for Synology's stock firmware, and I was hoping to stick with their OS, but if it's vastly and overwhelmingly better I could see switching to something else. Are snapshots actually necessary? Do they provide something incrementals to the external drive don't? I guess they're lighter weight and you can do them more often?
- ggm 2y ago
  If the FS implements snapshots "down low" you get COW efficient, and pretty much lock-free atomic "it's done" -if you run userspace "get me the difference" then it has all kinds of runtime costs and risks. That said, rsync is pretty good. Your J model can't get more memory. And you look to be right, its the OS it comes with and thats it. The NAS you have is the best NAS sometimes. It's all very well positing better ones, but they aren't the one you have: So make the most of it.
  - eternityforest 2y ago
    Yup, the best NAS is the one you can actually justify buying!