4 ms·
Sounds like the article should have been titled, "Why you shouldn't use ZFS". ZFS does not have such [recovery] tools, if the pool is corrupt, all data must
by jqpabc123 4y ago
Sounds like the article should have been titled, "Why you shouldn't use ZFS".
ZFS does not have such [recovery] tools, if the pool is corrupt, all data must
be considered lost, there is no option for recovery.
In other words, this marvelous piece of over engineered technology is fragile. When it works, it's great. When it fails, it's a complete disaster.
Everything fails eventually. How do you prefer your failure served? In small manageable increments or one spectacular, complete, overwhelming, unrecoverable helping.
- Dork1234 4y agoActually this is a good thing. If you have a failure you want to fix the faulty device and restore from backup. The biggest problem is having a failure, and then overwriting the good backup with failed data. I have had this happen to me, and have since switched to ZFS for storage I want to keep, with multiple backups.
- jqpabc123 4y agowith multiple backups Multiple rotating backups would have solved your problem before ZFS.
- Wowfunhappy 4y agoUnless you can somehow make backups every few seconds, restoring from a backup still means loosing data. It's just highly preferable to loosing all data. Also, on a non-checksummed filesystem, how will you know whether a file needs to be restored from the backup?
- freemint 4y ago> Also, on a non-checksummed filesystem, how will you know whether a file needs to be restored from the backup? Has same edit epoch? If yes skip. If no compute checksum of both and compare, copy over in case they differ. I am fully aware of the flaws of this.
- mustache_kimono 4y agoI think it's important to note this wasn't written by an expert on ZFS. And I'm pretty certain this author is actually wrong about this particular point, as ZFS does allow a number of extraordinary measures to recover a pool from a corrupted state, such as: zpool import -FX mypool, where: 1) -F Attempt rewind if necessary. 2) -X Turn on extreme rewind. 3) -T Specify a starting txg to use for import. There are a few other points you might consider: First, what kind of failure might cause an unrecoverable corrupted pool, and how useful are other filesystems data recovery tools actually, if corrupted in a similar fashion? Is there an apples to apples comparison that you can share? I believe the author and you are sharing speculation. Second, how do you know you have corruption with another filesystem until you read back the data? And do you even know you have corruption then?
- Avamander 4y agoSure, those options are nice, but they use the same code paths as regular reading/mounting does. If you've got some corruption that causes a crash, *all zfs tools are useless*. You must whip out some idiotic 13yo Python script to destroy uberblocks and you might make it work. Usually bug => corruption => module crashes => no zfs own tools work.
- mustache_kimono 4y agoFirst, the author is wrong by your own terms. If you think a 13yo Python script is the state of the art for ZFS recovery, that still doesn't comport with what the author said ("all data must be considered lost, there is no option for recovery"). The author is facially wrong. If you had said "I once had a corrupt pool I couldn't recover", that's a useful data point. But instead you tied your wagon to someone who obviously doesn't know much about what they are talking about. Second, you managed to ignore my two other original points -- 1) Is the state of other filesystems any better re: recovery in similar circumstances (show your work, when XFS is afflicted with the exact same type of corruption, how is it better)?, and 2) is it possible are you better positioned to deal with corruption with ZFS (because it's usually not silent)? I'm not saying ZFS is better for you. You are obviously not predisposed. What I am saying is -- your argument has to be better supported for it to make any sense.
- getcrunk 4y agoSeriously outside of very specific use cases I think there is a major bandwagoning with zfs. Using snapraid for larger unchanging data or normal raid without striping for vms or dbs allows your recovery story to be more flexible/robust
- mustache_kimono 4y ago> I think there is a major bandwagoning with zfs And I think there is a kooky caucus for half ass solutions. The Linux community has chosen to beclown itself by pretending ZFS isn't absolutely amazing (and free!), and the result is the strangest collection of system level kludges anywhere. You forgot Stratis and unraid!
- matheusmoreira 4y ago> there is a major bandwagoning with zfs Well, there is a huge number of reasons for that. Just recently ZFS got RAID-Z expansion capabilities. Before that, I would have had to make plans for a storage server and buy all the storage devices upfront. Now I can expand the storage pool as needed, one device at a time. The ability to do this is the only reason I even bothered with btrfs. Now that ZFS has flexible storage pool expansion, it is essentially perfect as far as I'm concerned.
- Wowfunhappy 4y agoIf you're at the point of using NTFS data recovery tools, you've already lost. With an enormous amount of time, effort, and/or money, you may be able to recover something, although without checksums you'll never be able to validate whatever-it-is you pull out. I wouldn't call it a "manageable" failure. The goal is to never reach this point, via a combination of redundancy and backups. ZFS helps enormously with the first part.
- crest 4y agosigh It still checksums everything it touches and can read damaged file systems in most cases. More importantly it can tell you which parts have been corrupted and which haven't. Because all live data on disk is checksummed and the checksum is part of the reference (except for the ring buffers of superblocks which carry their own checksums) you can find detect data corruption and recover to a sane state. If you detect and guestimate a fix for corruption on traditional file system by your own logic you would have to consider all file system data structures as well as all file content suspect with no recover path short of wiping the file system and restoring the data to a fresh file system. Oh wait your backup was made from an untrustworthy source... It's not as black and white as you or the blog post makes it out to be. I've had to recover damaged ZFS pools while traveling (there are neither affordable nor travel compatible no laptops with ECC RAM support). ZFS scrub told me which 4 files have been corrupted and overwriting them with good copies from an other machine solved the problem. Without ZFS (or a similar file system) I wouldn't have noticed the data corruption as quickly and wouldn't have known what to restore. Also ZFS is no one trick pony and has more to offer than "just" end to end checksumming e.g. pooled storage for multiple file systems and sparse block devices, fast consistent snapshots (good enough to backup a running RDBMs), incremental backups and replication (no more dreaded full backups), ease of administration, easy to grow capacity (as long as you do it in large enough increments), transparent compression and if you really need it block level deduplication (you probably don't want online block level dedup).
- hamandcheese 4y agoYou should have an offsite backup. If you don’t, you are asking for data loss, regardless of what filesystem you use. I personally am glad that ZFS doesn’t have “easy to partially recover a corrupted FS” on their list of priorities. Designing for that might take resources away from making the system as a whole more robust, and at best it’s a half-measure that will never be a substitute for an offsite backup solution.
- Avamander 4y ago> If you don’t, you are asking for data loss, regardless of what filesystem you use. Backups have a delay. A filesystem that fails catastrophically is significantly worse than the one that doesn't.
- hamandcheese 4y agoSuppose your FS has a 50% chance of partially recoverable failure in a given time range. Is that always better than the catastrophic alternative, even if the odds of catastrophic failure are 0.01%?
- Avamander 4y agoThat risk assessment depends on your use-case. Those percentages also aren't very correct. Assuming that new files are being constantly created and they're not of vital long-term integrity, the chance of preventing silent bit rot (ZFS is supposed to protect against) is worth fuck all if it's much more likely that a bug takes your pool irrecoverably with hours of data.
- rhinoceraptor 4y agoIt's just not true that if ZFS metadata is corrupted, then the entire pool is gone. ZFS is a transactional file system, so the pool will be put in a faulted state, and it gives you the option to roll back to before the faulty transaction. I really do not see how anyone could possibly think that is worse than your data being silently corrupted on disk, which then propagates to your backups, corrupting them too. People seem to think ZFS is worse because it tells you when your data is corrupted, rather than not telling you? It doesn't make any sense.
- louwrentius 4y agoIf you checksum garbled data from memory, that checksum is useless. Garbage in, garbage out.
- Wowfunhappy 4y agoYes, the checksum is useless, because the checksummed file is corrupt. By contrast, without checksumming... the file would still be corrupt! In the best case, when good data was written to disk, checksumming preserves the good data. In the worst case, when bad data was written to disk, checksumming does no harm. Checksumming is good.
- rhinoceraptor 4y agoNo one is saying that you shouldn't use ECC, obviously you should if at all possible. But even if you don't have it, that is no reason not to use ZFS.
- Avamander 4y ago> It's just not true that if ZFS metadata is corrupted, then the entire pool is gone. ZFS is a transactional file system, so the pool will be put in a faulted state, and it gives you the option to roll back to before the faulty transaction. It is true because ZFS on Linux crashes very quickly when metadata is corrupt. It can't roll back transactions because it'll crash almost instantly even looking at your pool, not to mention when scrubbing it.
- anonuser123456 4y agoYour file system is not a backup technique. All file systems will fail. The only time I’ve had ZFS fail, was when both drives in a mirror pool died within 8 minutes of another. No filesystem would save you there. Since ZFS has a replication protocol (ZFS send) recovery was basically a few ZFS revc calls into a new pool.
- yarg 4y agoAssuming that you bought two of the same device? https://news.ycombinator.com/item?id=32048148 https://news.ycombinator.com/item?id=32048148
- louwrentius 4y agoAuthor here: I think ZFS is totally fine, but it's about understanding risk. And running ZFS on your laptop without ECC, that's ok. Running ZFS on a home server as a NAS but without ECC memory, you are addressing one risk, but you are forgetting another (IMHO bigger) risk: faulty memory causing bitrot. It's as if you lock your back-door and leave the front-door unlocked.
- GekkePrutser 4y agoUnderstandable, but by telling people to just forget ZFS and use EXT4 instead, you're getting them to leave both doors open. Also, bitrot is not like burglars. Burglars try one door and if they don't succeed they will look for further vulnerabilities. Memory and disk corruption is random, it's not trying to attack you. In many cases ECC is just not possible without buying expensive new hardware (with possibly higher power consumption) due to intel being so precious with ECC as a 'server feature only'. Data protection is not absolute, protecting against one class of corruption is better than none. E.g. one of my NASs is a low-power NUC specifically chosen for its power consumption. Of course both is even better but not everyone has the financial resources for perfection.
- louwrentius 4y agoI would never tell people to forget ZFS, it’s understanding that ZFS alone just doesn’t reduce all bitrot risk. Groeten! In the end people worry too much, all consumer NAS gear doesn’t have any bitrot protection… (to put things in perspective).