19 ms·
Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take
by acd 6y ago
Insightful article. Zfs is the best file system. Ability to know that you do not have silent file corruption. Running without raid controllers. Feature to take snapshots super fast without waiting and that take little extra space. You can use Cache SSD with ZFS for read acceleration of physical disk. Transparent file compression. Good command line interface. Now you can take good actions with ZED for example sending a notification to an alert system like Slack, ticket system when disk fails or start disk scrubbing/rebuild.
If you are on Linux I can highly recommend ZFS and Minio for S3 like storage. ZFS for local storage.
- arianvanp 6y agoI don't understand where minio suddenly comes from? Minio is barely documented. I had to ask in Slack to interpret what it meant when minio said "your cluster is 5 red and 7 yellow" as the colours aren't event documented. Every minio cluster I hosted had dataloss. Each one has lead to a reported issue on their GitHub that hasn't been closed to date. Nothing about recovery is documented. Documentation is slim in general. Really confused why you're naming it next to ZFS which is sublimely documented and withstood the test of time. I'd advise something battletested through time like Ceph for object storage instead.
- wfn 6y agoCould you expand more re: your minio experience? We're about to enter production with minio as our S3 interface-providing file storage system, and have found documentation to be sufficient. However, I'm slightly concerned that we haven't tested it enough, and that doc and support may prove to be lacking (like in your case) when we hit edge cases and failure scenarios.
- ddorian43 6y agoI'm checking it out too. No LTS version. Bugs will be fixed in a new version, which may include new bugs. Some upgrades need full cluster restart. If you ask too many question you may get "it's better start a subscription".
- thr0w3345 6y agoI highly recommend you look at leofs instead — it’s been absolutely rock solid for us.
- seqizz 6y agoWe've been running for about a year now and had zero problems with it. (~20TB total, few single instances, serving as S3 backend to Restic - handmade HA - backup service)
- devonkim 6y agoI'm super confused how anyone could recommend Minio for the foundation of a production system as well. I've understood it to be primarily for development purposes and have used it for some local mock tests for S3 compatibility behind a firewall or to simulate error test cases more reliably in my applications. Even Ceph has its warts for distributed object storage as does basically... anything in existence worth considering that's OSS-ish (GlusterFS, HDFS, Lustre) but comparing Minio to ZFS is confusing given how drastically different in purpose, functionality, and engineering hardening has happened between the two projects. With that said, the AWS S3 team really is impressive in what they've built out and deserves more shout-outs from people outside Amazon.
- craigyk 6y agoIt's a bit on a non-sequitur, but I am also looking at using MinIO as a S3 interface to a ZFS filesystem. Would be interested to hear from others about this use case for MinIO and possible alternatives.
- diffeomorphism 6y agoCan you comment on btrfs by comparison? Also, what is the status of the license incompatibility/integration of ZFS with distros? Canonical seems to think that shipping it with ubuntu is legal, but other distros seem less sure.
- spindle 6y agoIt's trivial to use ZFS on NixOS, including on the root partition. The legal position is that ZFS is open source under a copyleft license but that many people think that it's illegal to bundle it with the Linux kernel because of some (I think unintended) incompatibilities between its license and the Linux kernel's license. Canonical (and some others) disagree, and think that it's legal. It's only the bundling that's at issue - everyone agrees that it's legal to use with Linux once you have both. See https://ubuntu.com/blog/zfs-licensing-and-linux https://ubuntu.com/blog/zfs-licensing-and-linux for Canonical's opinion.
- viraptor 6y agohttps://www.gnu.org/licenses/license-list.html#CDDL https://www.gnu.org/licenses/license-list.html#CDDL "This means a module covered by the GPL and a module covered by the CDDL cannot legally be linked together." I don't think the incompatibilities are really unintended. If anyone has enough lawyers and money to relicense something correctly, it's Oracle.
- trasz 6y agoExcept then you'd loose the patent protection provided by CDDL. And you'd cause licensing problems for literally everyone else but Linux. And you probably wouldn't gain anything in the long run anyway; AdvFS was released under GPL and went nowhere.
- capitol_ 6y agoThe incompatibility was very much intended, Sun needed a way to compete with linux and didn't want to be assimilated into the linux ecosystem, so they released opensolaris under CDDL instead of GPL or MIT. Oracle haven't re-licensed it for their own reasons. Here is some more info on the subject: https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/ https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/
- DonHopkins 6y agoThat's a big checklist of features, but can it murder your wife? https://en.wikipedia.org/w/index.php?title=Comparison_of_file_systems&oldid=209063556#Features https://en.wikipedia.org/w/index.php?title=Comparison_of_fil...
- louwrentius 6y agoI would advice people to think about what they need. ZFS is fine, but it is overkill for most home applications and it has a pitfall related to extensibility. https://louwrentius.com/what-home-nas-builders-should-understand-about-silent-data-corruption.html https://louwrentius.com/what-home-nas-builders-should-unders... https://louwrentius.com/the-hidden-cost-of-using-zfs-for-your-home-nas.html https://louwrentius.com/the-hidden-cost-of-using-zfs-for-you... So it really depends on your needs. Statements as "ZFS is the best filesystem" are so meaningless. P.S. SSD caching often has no tangible practical benefit for most applications. More RAM is often the better investment. As is with any filesystem.
- m0zg 6y ago> ZFS is fine, but it is overkill for most home applications Yeah, not having silent data corruption is "overkill", sure. /s Why not use ZFS? It takes 15 seconds to install, and its CLI is fairly intuitive. Works fine. Costs $0. Why not, even for "home" applications? I could see how it could be unsuitable for "entreprise" applications where there are strict performance requirements etc, but for home, I wish I could use ZFS everywhere.
- paulie_a 6y agoIf it's cli it's a nonstarter Even if it has a gui it's probably a non starter unless there are literally 2 options
- StavrosK 6y agoFor one, because of the fact that you can't add an extra disk to your pool if you need to. GP is saying "plan better", no need to get snarky.
- magnetic 6y agoWhat do you mean? I've done this not so long ago on my pool. https://unix.stackexchange.com/questions/530968/adding-disks-to-zfs-pool https://unix.stackexchange.com/questions/530968/adding-disks...
- StavrosK 6y agoI have a home NAS that has an SSD on it for the OS and four HDDs in RAIDZ. Does anyone know if/how I can use a small part/partition of the SSD for the cache? I don't need an entire SSD's worth of cache, and I'd rather not have to buy an extra one.
- lscotte 6y agoYes, you can. It's called L2ARC.
- StavrosK 6y agoThanks for the tip, I'll look into it and deploy it!
- louwrentius 6y agoI would really advise not to use the SSD for cache: you'll only wear it out and do you have any evidence that you benefit from it?
- StavrosK 6y agoProbably not, since it's a home NAS the reads won't be very repeatable. I wanted to prevent the hard disks from waking up, but it's mostly writes, so it doesn't help there either.
- washadjeffmad 6y agoAdd the SSD to its own pool, set up as a working share, then use a utility or script to sync contents a few times a day?
- StavrosK 6y agoThat would probably be easier, yep, thanks. It's more straightforward to just make a partition on it and rsync twice a day.
- 6y ago
- jiveturkey 6y ago> Zfs is the best file system. Of course it isn't. There can't be a single best filesystem. For the use case of zfs, yes it is the best filesystem.