Jul 2 12:51:02 <kern.crit> newcoruscant kernel: ahcich1: AHCI reset: device not ready after 31000ms (tfd = 00000080) Looks like the FreeBSD kernel has a simil

Jul 2 12:51:02 <kern.crit> newcoruscant kernel: ahcich1: AHCI reset: device not ready after 31000ms (tfd = 00000080) Looks like the FreeBSD kernel has a similar configuration as Linux, which is a command timer of ~30 seconds. If the drive hangs longer, then the whole link is reset. And the reason why the drive is hanging is lost in that reset. The likely reason why the drive is hanging, if it's a consumer drive, is it has very high bad sector recovery time, and can approach 3 minutes. That's pretty crazy. Anyway, it's central to any RAID to be able to get a discrete read error from the drive. And that read error will include the LBA for the bad sector. And that information is needed to know what data is affected and where to get a good copy (from mirror or from reconstruction using parity). This is the same on md raid on Linux, ZFS, Btrfs, and even hardware RAID will depend on it. There are too many commands in the queue to just assume one of those commands and therefore one of those requested sectors (which could be thousands) is the bad sector. A discrete read error with LBA is necessary. And the link reset prevents that. On Linux, this is set per drive (this is a Linux SCSI command timer, which applies to all PATA and SATA drives too, and it is not a drive setting, it's a kernel setting) timeout: $ cat /sys/block/sda/device/timeout 30 Ideally the drive supports SCT ERC (that's for SATA, there's a SCSI/SAS equivalent) and you use 'smartctl -l scterc' to change it to a value less than the kernel command timer. That way the drive itself gives up faster, and issues a discrete read error, and now the kernel (and ZFS) can figure out how to repair the problem. If the drive doesn't support configurable SCT ERC, you have to raise the kernel command timer to an obscene level. # echo 180 > /dev/block/sdN/device/timeout And now the kernel will just wait and wait and wait and wait until finally the drive does give up on recovery and issues a discrete read error, and the kernel and ZFS can fix it. The late tl;dr is, this is not a ZFS problem. This is the result of long set kernel command timers, and a refusal by kernel developers to update them for the incredibly high (some might say crazy) deep recoveries that consumer drives use. But here's the thing, macOS and Windows know about these high recoveries, and tolerate them without doing link resets. And that's why they eventually recover bad sectors, but the user notices them as performance slow downs. And what fixes them? A clean install. And that's because a sector write that fails due to a bad sector, causes a remap. Which is the same mechanism that ZFS, Btrfs, md raid, and hardware raid will all depend on. The read error results in getting good data from a copy (mirror or reconstruction from parity), the good copy is both sent to the application layer as well as results in an overwrite command to the sector that had the read error. If that write fails the drive firmware itself remaps that LBA to a spare sector. Anyway, this is a misconfiguration, the question is who is to blame for it? And even that's complicated because the drive doesn't announce its SCT ERC support or value. You have to poll it. Could the kernel poll for this? I guess? But should it? Probably not its domain? Is it a distribution question rather than an upstream kernel question? Perhaps, yes, the distros should use high command timers by default, and expect sysadmins will reduce them if the use case requires it.