6 ms·

No, ZFS really doesn't need an fsck tool

112 points by antonios 14y ago

PaulHoule 14y ago
Nope, it just needs to stop having wrecks.
spikels 14y ago
Maybe I'm just tired this morning but I wish this article could just get to the point. I feel like I'm reading a mystery novel but I am never going to make it to the end and find out who did it.
- 4ad 14y ago
  This article explains in-depth why ZFS doesn't need a fsck tool. It's very much to the point in its entirety. I'm sorry content-free articles posted here lately have diluted your expectations.
  - spikels 14y ago
    I agree this article in not content free but all these points could be made in many fewer and less convoluted words. And they buried the real news (at least to me) that the hardware is lying to the OS. Did not mean to offend.
    - andrewflnr 14y ago
      It's in the first sentence of the fourth paragraph.
- MereInterest 14y ago
  It's right there in the title. ZFS does not need a fsck tool. That is the point. All the rest of it is telling you why.
- inopinatus 14y ago
  tl;dr version: ZFS doesn't need fsck because it has virtual log-structured metadata and can therefore always recover itself.
brianlouisw 14y ago
love the domain name
joosters 14y ago
The article doesn't ever consider that ZFS might have bugs. Dodgy disks, bad firmware, power failures, yes. But no consideration that the ZFS code could contain problems. If you are happy that the ZFS code is perfect, then it makes sense to rely upon its consistency checks, snapshot features, etc (and I'm not criticising those). But what if ZFS isn't 100%? How do you recover your data?
- Breakthrough 14y ago
  Even ZFS isn't a replacement for backups. Think of it more as a better indicator of when things are happening that clearly shouldn't be (e.g. hardware problems).
- 4ad 14y ago
  Any filesystem might have bugs and you can't rely on fsck to alleviate that problem (in fact it might make it worse). That problem is solved by having backups. ZFS dis not a replacement for backups. Oh, and there's always zdb, the ZFS debugger, you can use it to walk the on-disk structures.
  - tjoff 14y ago
    The fsck can't make it worse if you mirror the drive first. Also, is "you can't rely on fsck to alleviate that problem" an argument for not having an fsck?
- jiggy2011 14y ago
  Surely you run that risk with any FS? What about the recovery tools having bugs?
  - joosters 14y ago
    True and true. But a decent checker like e2fsck should be paranoid & untrusting of on-disk data structures, and able to retrieve at least some data from a badly-mangled filesystem. It is a fallback tool but a useful one.
ChuckMcM 14y ago
Interesting rant (from 2009). At NetApp the WAFL file system is also always consistent on disk, so it too doesn't need fsck. That said, WAFL had 'wack' (WAfl ChecK) which could go through and check that the on disk image was correct. Unlike UFS or FFS or EXTn the file system couldn't be corrupted by loss of power mid write, but like ZFS it can be corrupted by bugs in the code which write a corrupted version to disk. So the tool does something similar to fsck but it is simpler, more of a data structure check rather than a "recreate the flow of buffers through the buffer cache to save as much as possible" exercise.
- X-Istence 14y ago
  zpool scrub
- thrownaway2424 14y ago
  Indeed. My story from 1999. Worked at a company that bought a ton of NetApp filers to support a webmail service. NetApp sales engineers swear on their mothers' graves that there is no such thing as fsck for wafl, that wafl always transitions from one gold-plated consistent state to the next with no possibility of metadata inconsistency. OK. Three months later, big outage. On-site techs report the filers display "fast wack" on the front panel. Call NetApp support. What is "fast wack"? That's the fsck. Assholes! It turned out that the filer had got corrupt somehow, and wack itself could not comprehend a filesystem with more than 2 billion files. Inode number stored in signed int32. Major, major surgery, hotpatching of filer firmware, three days of downtime, serious negative press coverage. Bottom line: whenever anyone tells you their filesystem is guaranteed to be consistent, kick that person right in the shins.
- gnosis 14y ago
  "At NetApp the WAFL file system is also always consistent on disk, so it too doesn't need fsck." How does it manage to stay consistent if a cosmic ray strikes it and flips one or more bits? How does it manage to stay consistent if you physically bump in to the drives and cause physical damage by having the disk head briefly touch the disk surface? Wouldn't you need a filesystem consistency check and repair tool like fsck in these cases?
abtinf 14y ago
ZFS may not need fsck, but it would be great if Oracle would re-open-source it. I've considered using it, but I can't trust that it has a future. I'm also rather confused by Oracle contributing to btrfs while also building ZFS privately. My intuition is that if they open-sourced ZFS and offered it under a dual BSD/GPL license, it would become the fs standard overnight.
- antonios 14y ago
  It has a future. FreeBSD and companies based on it like IXSystems have adopted it and developers are actively hacking it. In FreeBSD version 10 it will feature TRIM support as well as a new data compression scheme (LZ4): https://wiki.freebsd.org/WhatsNew/FreeBSD10 https://wiki.freebsd.org/WhatsNew/FreeBSD10
- X-Istence 14y ago
  ZFS is open sourced, and it is available in FreeBSD for example. Most of the development has been going on in Illumos and the team behind that are some of the original heavy hitters that worked on ZFS.
- profquail 14y ago
  Why use a dual BSD/GPL license? I think that would inevitably lead to a fork between the two versions -- for something like ZFS that's going to be used on many different platforms and which is already well-established, the Apache 2.0 license probably makes the most sense.
ianlevesque 14y ago
So if you have an unmountable zfs pool, instead of reaching for fsck (which doesn't and won't exist) you can instead do: zpool clear -F data And it will take advantage of the copy-on-write nature of ZFS to roll back to the last verifiably consistent state of your filesystem. That's a lot better than the uncertain fixes applied by fsck to other file systems. It even tells you how old the restored state is.
- cbr 14y ago
  Why not ship with fsck.zfs="zpool clear -F data"? Then people would stop complaining.
blinkingled 14y ago
Been running it since 0.6.0-rc14 on a Proliant Microserver with ECC RAM and I am happy with it. 4x2TB RAIDZ internal, and 2x1TB USB3 zpools with SSD for zil and logs. Shared over GigE using Samba4 and AFP. Performance is decent enough with lz4 compression and dedup off. Dedup on takes more CPU but nothing even the 2.2Ghz Turion can't handle. Main thing is stability has improved a lot too. If you want the utmost performance may be this isn't for you but for NAS/backup/streaming type usage ZFS on Linux is nearly perfect.
c0t0d0s0 14y ago
1. When there is a bug in the code that writes the ZFS stuff, why should the bug be addressed by the fsck code. This would assume, that you know of the bug beforehand, but then you could better fix the bug in the code that writes. 2. When there is a bug in the on-disk-state it should be addressed by the code that reads the data , not by a fsck tool. 2.1. The correction of the bug in the on-disk-state should be done on the basis of the exact knowledge about the bug and not by a generic check tool. 3. Repair is always based on assumptions. Those could be correct or incorrect. The more you know about the problem that led to the repair-worthy state, the more probable the assumptions are correct. 4. What is the reasoning behind the argument "when your metadata is corrupt , that the data is correct" and so you could repair metadata corruption without problems. It sounds more sensible to fall back to the last known correct and consistent state of metadata and data, based on the on-disk-state represented by the pointer structure of the ueberblock with the highest transaction group commit number with a correct checksum . The Transaction Group rollback at mount does exactly this.
deelowe 14y ago
The whole fsck discussion seems baffling to me. While I'm no ZFS expert, I've been using it for several years now and my understanding is this. Take what a normal fsck type tool does and build those features into the underlying FS and supporting toolchain. For what ZFS does and how it works, it really doesn't make sense to me at all for it to have an "fsck," whatever that means. Really, it's hard to even imagine what an "fsck" would do for zfs. You'd just end up rewriting bits of the toolchain or asking for the impossible. I asked this in the other thread, but I'll ask here again. Excluding semantics, what is it that people want fsck to do specifically that zfs doesn't provide a method for already? Seriously, the question to me seems akin to asking why manufacturers don't publish the rpm spec for SSDs. It's a really odd thing to ask and can't be answered without an exhaustive review of the mechanics of the system. I can't help but get the feeling that a lot of people complaining about ZFS have very little knowledge or familiarity with it and/or BSD/Unix in general. ZFS is not like any Linux FS. It doesn't use fstab, the toolchain is totally different, the FS is fundamentally different. It was built for Solaris and really reflects their ideology, which is completely foreign to people who only have familiarity with Linux. Accept it and move on or don't, but I've yet to see any evidence to back up these claims other than "this is what is done in Linux for everything else" which is just FUD.
- tomku 14y ago
  For most people, fsck and filesystems that rely on fsck are a known quantity. They understand that a filesystem does what it can to keep itself consistent, but that sometimes outside assistance is necessary in the form of fsck. When you show them ZFS and say "Oh, it doesn't need a fsck program", they assume that you're bullshitting them to cover up for the fact that it doesn't have a fsck program yet. As far as the Linux/Unix thing, I think you're reading way too far into it. Linux neither invented nor popularized filesystem consistency checking programs. Unix filesystems (such as UFS) often have consistency checking programs as well, though they might not be called "fsck". Windows has Scandisk. ZFS is the odd one out here, and it's not surprising for people to treat it as such. Give it time, and if ZFS's approach becomes more widespread, people will come around. tl;dr: It's about trust, not ignorance.
  - 14y ago
jiggy2011 14y ago
This reminds me of when I introduce people to Linux and they insist that there must really be a C drive.
ScottBurson 14y ago
I lost a ZFS pool once. The cause ultimately turned out to be a slowly failing PSU. (It was an expensive OCZ PSU, too, which is why I didn't suspect it as quickly as I probably should have. OCZ did replace it under warranty without argument.) It was a development machine, so it wasn't being backed up. I thought it was just one disk going bad; by the time it was clear that it was something worse than that, it was too late. Most of the important contents of the pool had been checked into the VCS, but not everything. I wound up grepping the raw disk devices to find the latest versions of a couple of files. Any filesystem would have had serious trouble in such a situation, of course. But I can't help thinking that picking up the pieces might have been easier with, say, EXT3. On the other hand, I think it speaks well for ZFS that a slowly failing PSU seems to be almost the only way to lose a pool.
DiabloD3 14y ago
I don't understand why people think ZFS doens't have a fsck tool: zpool scrub
- Karunamon 14y ago
  Probably because on at least two distros, fsck.zfs is just a symlink to a bash script that runs "exit 0". That kind of threw me for a loop.
  - DiabloD3 14y ago
    Thats to deal with the fact their init scripts blindly call "fsck.fstype", and zfs doesn't have a fsck nor needs one.
nwf 14y ago
A slight disagreement: the advantage of a ZFS online consistency checker would be to help ensure that there are no bugs in ZFS. It appears that ZFS lacks a full consistency checker -- scrub only walks the tree and computes checksums; notably absent in this procedure appears to be validating the DDT. While ZFS claims to be always on-disk consistent--and I certainly believe that the intent is that it be so!--I seem to have tripped over some bug ( http://lists.freebsd.org/pipermail/freebsd-fs/2013-March/016627.html http://lists.freebsd.org/pipermail/freebsd-fs/2013-March/016... ) which corrupted the DDT, and now I have no way of rebuilding it, so I dropped $$$ (for me) on a new disk array and zfs send | zfs recv so that everything rebuilt. That's sort of crazy, if I may be so bold. I suppose I could take the pool offline for several days and poke at it with zdb, but that is not really desirable either.
derleth 14y ago
It reminds me of how people used to think all filesystems needed to be explicitly defragmented because of design flaws in FAT, which was designed for floppies (and wasn't especially well-designed even at that). http://geekblog.oneandoneis2.org/index.php/2006/08/17/why_doesn_t_linux_need_defragmenting http://geekblog.oneandoneis2.org/index.php/2006/08/17/why_do... http://www.howtogeek.com/115229/htg-explains-why-linux-doesnt-need-defragmenting/ http://www.howtogeek.com/115229/htg-explains-why-linux-doesn...
jodrellblank 14y ago
And after a dozen paragraphs on how ZFS is unlikely to get corrupted, the meat of the conent: "my opinion is that you shouldn't try to repair it anyway". Anyway: You do not repair the state last state of the data. And in my opinion: You should not try to repair it ... at least not by automatic means. Such a repair would be risky in any case. [..] In this situation i would just take my money from the table and call it a day. You may lose the last few changes, but your tapes are older. This "you do not need an emergency repair tool because in an emergency I think you should just forget it" is exactly the claim that this blog post was supposed to be countering. Explaining why a do-the-best-you-can repair utility is not necessary, and the argument it boils down to is "because I don't think you should do that".
- c0t0d0s0 14y ago
  The basic problem of filesystem repair is the point that you repair the metadata, but it can do nothing about your data. So when your filesystem enables you to fall back to a known consistent state of metadata and data by the COW, you should fall back to a known consistent state. And not to something that is repaired by a generic tool. How do you know in such a situation that somewhere in you thousands and thousands of files the repair got something wrong. The copy on write behaviour of ZFS has it's advantages. And as i already wrote in a different comment: If there is a bug in the stuff writing the on-disk state, the bug should be addressed on the exact knowledge of the bug in the code reading the on-disk-state and thus doesn't make assumption what could be halfway correct, but by some piece of code that does the correct with the incorrect on-disk-state.
  - jodrellblank 14y ago
    Yes, I know you said both those two things, and I agree - ZFS has inbuilt error detection and healing. Any code that can detect errors and heal them should be there. And if you have corruption the only long term, safe, ass-covering advice to give is to restore to a pre-corrupt state. But the argument went: Detractor: ZFS needs fsck. You: No it doesn't. Detractor: ZFS creators attitude has always been "we don't think it should exist", but there's no more reason than this. It still corrupts so it still needs an fsck tool. You: Here is a big blog post about why it doesn't: OK so it can get corrupted but I don't think an fsck tool should exist. You know how useful it is to post on StackExchange "help I have this situation, I know conceptually there is a way out, but how can I actually do it?" and get the replies "you shouldn't want to do that"? It's not helpful at all.

112 points by antonios 14y ago

Nope, it just needs to stop having wrecks.

Maybe I'm just tired this morning but I wish this article could just get to the point. I feel like I'm reading a mystery novel but I am never going to make it to the end and find out who did it.

4ad 14y ago
This article explains in-depth why ZFS doesn't need a fsck tool. It's very much to the point in its entirety. I'm sorry content-free articles posted here lately have diluted your expectations.
- spikels 14y ago
  I agree this article in not content free but all these points could be made in many fewer and less convoluted words. And they buried the real news (at least to me) that the hardware is lying to the OS. Did not mean to offend.
  - andrewflnr 14y ago
    It's in the first sentence of the fourth paragraph.
MereInterest 14y ago
It's right there in the title. ZFS does not need a fsck tool. That is the point. All the rest of it is telling you why.
inopinatus 14y ago
tl;dr version: ZFS doesn't need fsck because it has virtual log-structured metadata and can therefore always recover itself.

brianlouisw 14y ago

love the domain name

joosters 14y ago

The article doesn't ever consider that ZFS might have bugs. Dodgy disks, bad firmware, power failures, yes. But no consideration that the ZFS code could contain problems. If you are happy that the ZFS code is perfect, then it makes sense to rely upon its consistency checks, snapshot features, etc (and I'm not criticising those). But what if ZFS isn't 100%? How do you recover your data?

Breakthrough 14y ago
Even ZFS isn't a replacement for backups. Think of it more as a better indicator of when things are happening that clearly shouldn't be (e.g. hardware problems).
4ad 14y ago
Any filesystem might have bugs and you can't rely on fsck to alleviate that problem (in fact it might make it worse). That problem is solved by having backups. ZFS dis not a replacement for backups. Oh, and there's always zdb, the ZFS debugger, you can use it to walk the on-disk structures.
- tjoff 14y ago
  The fsck can't make it worse if you mirror the drive first. Also, is "you can't rely on fsck to alleviate that problem" an argument for not having an fsck?
jiggy2011 14y ago
Surely you run that risk with any FS? What about the recovery tools having bugs?
- joosters 14y ago
  True and true. But a decent checker like e2fsck should be paranoid & untrusting of on-disk data structures, and able to retrieve at least some data from a badly-mangled filesystem. It is a fallback tool but a useful one.

ChuckMcM 14y ago

Interesting rant (from 2009). At NetApp the WAFL file system is also always consistent on disk, so it too doesn't need fsck. That said, WAFL had 'wack' (WAfl ChecK) which could go through and check that the on disk image was correct. Unlike UFS or FFS or EXTn the file system couldn't be corrupted by loss of power mid write, but like ZFS it can be corrupted by bugs in the code which write a corrupted version to disk. So the tool does something similar to fsck but it is simpler, more of a data structure check rather than a "recreate the flow of buffers through the buffer cache to save as much as possible" exercise.

X-Istence 14y ago
zpool scrub
thrownaway2424 14y ago
Indeed. My story from 1999. Worked at a company that bought a ton of NetApp filers to support a webmail service. NetApp sales engineers swear on their mothers' graves that there is no such thing as fsck for wafl, that wafl always transitions from one gold-plated consistent state to the next with no possibility of metadata inconsistency. OK. Three months later, big outage. On-site techs report the filers display "fast wack" on the front panel. Call NetApp support. What is "fast wack"? That's the fsck. Assholes! It turned out that the filer had got corrupt somehow, and wack itself could not comprehend a filesystem with more than 2 billion files. Inode number stored in signed int32. Major, major surgery, hotpatching of filer firmware, three days of downtime, serious negative press coverage. Bottom line: whenever anyone tells you their filesystem is guaranteed to be consistent, kick that person right in the shins.
gnosis 14y ago
"At NetApp the WAFL file system is also always consistent on disk, so it too doesn't need fsck." How does it manage to stay consistent if a cosmic ray strikes it and flips one or more bits? How does it manage to stay consistent if you physically bump in to the drives and cause physical damage by having the disk head briefly touch the disk surface? Wouldn't you need a filesystem consistency check and repair tool like fsck in these cases?

abtinf 14y ago

ZFS may not need fsck, but it would be great if Oracle would re-open-source it. I've considered using it, but I can't trust that it has a future. I'm also rather confused by Oracle contributing to btrfs while also building ZFS privately. My intuition is that if they open-sourced ZFS and offered it under a dual BSD/GPL license, it would become the fs standard overnight.

antonios 14y ago
It has a future. FreeBSD and companies based on it like IXSystems have adopted it and developers are actively hacking it. In FreeBSD version 10 it will feature TRIM support as well as a new data compression scheme (LZ4): https://wiki.freebsd.org/WhatsNew/FreeBSD10 https://wiki.freebsd.org/WhatsNew/FreeBSD10
X-Istence 14y ago
ZFS is open sourced, and it is available in FreeBSD for example. Most of the development has been going on in Illumos and the team behind that are some of the original heavy hitters that worked on ZFS.
profquail 14y ago
Why use a dual BSD/GPL license? I think that would inevitably lead to a fork between the two versions -- for something like ZFS that's going to be used on many different platforms and which is already well-established, the Apache 2.0 license probably makes the most sense.

ianlevesque 14y ago

So if you have an unmountable zfs pool, instead of reaching for fsck (which doesn't and won't exist) you can instead do: zpool clear -F data And it will take advantage of the copy-on-write nature of ZFS to roll back to the last verifiably consistent state of your filesystem. That's a lot better than the uncertain fixes applied by fsck to other file systems. It even tells you how old the restored state is.

cbr 14y ago
Why not ship with fsck.zfs="zpool clear -F data"? Then people would stop complaining.

blinkingled 14y ago

Been running it since 0.6.0-rc14 on a Proliant Microserver with ECC RAM and I am happy with it. 4x2TB RAIDZ internal, and 2x1TB USB3 zpools with SSD for zil and logs. Shared over GigE using Samba4 and AFP. Performance is decent enough with lz4 compression and dedup off. Dedup on takes more CPU but nothing even the 2.2Ghz Turion can't handle. Main thing is stability has improved a lot too. If you want the utmost performance may be this isn't for you but for NAS/backup/streaming type usage ZFS on Linux is nearly perfect.

c0t0d0s0 14y ago

1. When there is a bug in the code that writes the ZFS stuff, why should the bug be addressed by the fsck code. This would assume, that you know of the bug beforehand, but then you could better fix the bug in the code that writes. 2. When there is a bug in the on-disk-state it should be addressed by the code that reads the data , not by a fsck tool. 2.1. The correction of the bug in the on-disk-state should be done on the basis of the exact knowledge about the bug and not by a generic check tool. 3. Repair is always based on assumptions. Those could be correct or incorrect. The more you know about the problem that led to the repair-worthy state, the more probable the assumptions are correct. 4. What is the reasoning behind the argument "when your metadata is corrupt , that the data is correct" and so you could repair metadata corruption without problems. It sounds more sensible to fall back to the last known correct and consistent state of metadata and data, based on the on-disk-state represented by the pointer structure of the ueberblock with the highest transaction group commit number with a correct checksum . The Transaction Group rollback at mount does exactly this.

deelowe 14y ago

The whole fsck discussion seems baffling to me. While I'm no ZFS expert, I've been using it for several years now and my understanding is this. Take what a normal fsck type tool does and build those features into the underlying FS and supporting toolchain. For what ZFS does and how it works, it really doesn't make sense to me at all for it to have an "fsck," whatever that means. Really, it's hard to even imagine what an "fsck" would do for zfs. You'd just end up rewriting bits of the toolchain or asking for the impossible. I asked this in the other thread, but I'll ask here again. Excluding semantics, what is it that people want fsck to do specifically that zfs doesn't provide a method for already? Seriously, the question to me seems akin to asking why manufacturers don't publish the rpm spec for SSDs. It's a really odd thing to ask and can't be answered without an exhaustive review of the mechanics of the system. I can't help but get the feeling that a lot of people complaining about ZFS have very little knowledge or familiarity with it and/or BSD/Unix in general. ZFS is not like any Linux FS. It doesn't use fstab, the toolchain is totally different, the FS is fundamentally different. It was built for Solaris and really reflects their ideology, which is completely foreign to people who only have familiarity with Linux. Accept it and move on or don't, but I've yet to see any evidence to back up these claims other than "this is what is done in Linux for everything else" which is just FUD.

tomku 14y ago
For most people, fsck and filesystems that rely on fsck are a known quantity. They understand that a filesystem does what it can to keep itself consistent, but that sometimes outside assistance is necessary in the form of fsck. When you show them ZFS and say "Oh, it doesn't need a fsck program", they assume that you're bullshitting them to cover up for the fact that it doesn't have a fsck program yet. As far as the Linux/Unix thing, I think you're reading way too far into it. Linux neither invented nor popularized filesystem consistency checking programs. Unix filesystems (such as UFS) often have consistency checking programs as well, though they might not be called "fsck". Windows has Scandisk. ZFS is the odd one out here, and it's not surprising for people to treat it as such. Give it time, and if ZFS's approach becomes more widespread, people will come around. tl;dr: It's about trust, not ignorance.
- 14y ago

jiggy2011 14y ago

This reminds me of when I introduce people to Linux and they insist that there must really be a C drive.

ScottBurson 14y ago

I lost a ZFS pool once. The cause ultimately turned out to be a slowly failing PSU. (It was an expensive OCZ PSU, too, which is why I didn't suspect it as quickly as I probably should have. OCZ did replace it under warranty without argument.) It was a development machine, so it wasn't being backed up. I thought it was just one disk going bad; by the time it was clear that it was something worse than that, it was too late. Most of the important contents of the pool had been checked into the VCS, but not everything. I wound up grepping the raw disk devices to find the latest versions of a couple of files. Any filesystem would have had serious trouble in such a situation, of course. But I can't help thinking that picking up the pieces might have been easier with, say, EXT3. On the other hand, I think it speaks well for ZFS that a slowly failing PSU seems to be almost the only way to lose a pool.

DiabloD3 14y ago

I don't understand why people think ZFS doens't have a fsck tool: zpool scrub

Karunamon 14y ago
Probably because on at least two distros, fsck.zfs is just a symlink to a bash script that runs "exit 0". That kind of threw me for a loop.
- DiabloD3 14y ago
  Thats to deal with the fact their init scripts blindly call "fsck.fstype", and zfs doesn't have a fsck nor needs one.

nwf 14y ago

A slight disagreement: the advantage of a ZFS online consistency checker would be to help ensure that there are no bugs in ZFS. It appears that ZFS lacks a full consistency checker -- scrub only walks the tree and computes checksums; notably absent in this procedure appears to be validating the DDT. While ZFS claims to be always on-disk consistent--and I certainly believe that the intent is that it be so!--I seem to have tripped over some bug ( http://lists.freebsd.org/pipermail/freebsd-fs/2013-March/016627.html http://lists.freebsd.org/pipermail/freebsd-fs/2013-March/016... ) which corrupted the DDT, and now I have no way of rebuilding it, so I dropped $$$ (for me) on a new disk array and zfs send | zfs recv so that everything rebuilt. That's sort of crazy, if I may be so bold. I suppose I could take the pool offline for several days and poke at it with zdb, but that is not really desirable either.

derleth 14y ago

It reminds me of how people used to think all filesystems needed to be explicitly defragmented because of design flaws in FAT, which was designed for floppies (and wasn't especially well-designed even at that). http://geekblog.oneandoneis2.org/index.php/2006/08/17/why_doesn_t_linux_need_defragmenting http://geekblog.oneandoneis2.org/index.php/2006/08/17/why_do... http://www.howtogeek.com/115229/htg-explains-why-linux-doesnt-need-defragmenting/ http://www.howtogeek.com/115229/htg-explains-why-linux-doesn...

jodrellblank 14y ago

And after a dozen paragraphs on how ZFS is unlikely to get corrupted, the meat of the conent: "my opinion is that you shouldn't try to repair it anyway". Anyway: You do not repair the state last state of the data. And in my opinion: You should not try to repair it ... at least not by automatic means. Such a repair would be risky in any case. [..] In this situation i would just take my money from the table and call it a day. You may lose the last few changes, but your tapes are older. This "you do not need an emergency repair tool because in an emergency I think you should just forget it" is exactly the claim that this blog post was supposed to be countering. Explaining why a do-the-best-you-can repair utility is not necessary, and the argument it boils down to is "because I don't think you should do that".

c0t0d0s0 14y ago
The basic problem of filesystem repair is the point that you repair the metadata, but it can do nothing about your data. So when your filesystem enables you to fall back to a known consistent state of metadata and data by the COW, you should fall back to a known consistent state. And not to something that is repaired by a generic tool. How do you know in such a situation that somewhere in you thousands and thousands of files the repair got something wrong. The copy on write behaviour of ZFS has it's advantages. And as i already wrote in a different comment: If there is a bug in the stuff writing the on-disk state, the bug should be addressed on the exact knowledge of the bug in the code reading the on-disk-state and thus doesn't make assumption what could be halfway correct, but by some piece of code that does the correct with the incorrect on-disk-state.
- jodrellblank 14y ago
  Yes, I know you said both those two things, and I agree - ZFS has inbuilt error detection and healing. Any code that can detect errors and heal them should be there. And if you have corruption the only long term, safe, ass-covering advice to give is to restore to a pre-corrupt state. But the argument went: Detractor: ZFS needs fsck. You: No it doesn't. Detractor: ZFS creators attitude has always been "we don't think it should exist", but there's no more reason than this. It still corrupts so it still needs an fsck tool. You: Here is a big blog post about why it doesn't: OK so it can get corrupted but I don't think an fsck tool should exist. You know how useful it is to post on StackExchange "help I have this situation, I know conceptually there is a way out, but how can I actually do it?" and get the replies "you shouldn't want to do that"? It's not helpful at all.