Argh, I got bit by a similar issue a while ago when I was expanding a zpool. The pool was originally created using 512b drives. After in-place subbing each lowe

Argh, I got bit by a similar issue a while ago when I was expanding a zpool. The pool was originally created using 512b drives. After in-place subbing each lower capacity old drive for a new, bigger (and also 4k) drive and resilvering, I realized that the zpool was running with emulated 512b sectors. Since freebsd zfs doesn't let you modify ashift after pool creation, I was stuck. However, I had some benchmarks laying around and was able to determine that the perf hit wasn't that bad for my workflow (<10%). So, I figure I'll just live with it until I eventually have enough disk to create a new pool with correct ashift and be able to copy all the data on the missized pool without taking anything offline/going under-protection. Definitely not the sort of problem I expected to encounter during that project, though, so I can totally see how even you guys would hit stuff like that.