4 ms·

Yup, and as far as "blackbox" solutions go, if something horrible should happen to your system, you want something that works like ZFS as opposed to, say, some

by dkh 5y ago

Yup, and as far as "blackbox" solutions go, if something horrible should happen to your system, you want something that works like ZFS as opposed to, say, some high-performance but proprietary hardware RAID. As someone who has a couple times needed to rip 100tb of ZFS disks out of something, put them into a different machine with a different architecture or OS in some random order, and access everything without having lost anything, it's hard for me to overstate how great it is that such a thing is even possible.

pdimitar 5y ago
How exactly did you do that? I'm learning ZFS still and I'm planning to move some of the disks in my first NAS to a better machine. Which ZFS commands?
- Filligree 5y ago
  Nothing special is necessary. All you need to do is import the pool as normal. There’s a potential problem, in that you can’t import a poll on an OS version that lacks the feature flags that are enabled on the pool. The way to solve that is to choose a common subset when creating the pool.
  - sennight 5y ago
    Unless they fixed it, another edge case is a change in processor endianness. I started working on a patch a long time ago, but it necessitated changes to code that had remained unchanged since the original release... not a blamelog I'm totally comfortable with.
    - Filligree 5y ago
      I hadn’t heard of that one. Does send |receive still work?
      - sennight 5y ago
        Nope, the system would instantly panic - presumably in order to protect data integrity... imagine the damage possible otherwise. It was actually a fairly ponderous debugging effort, as I remember it, because it was occurring on an already shaky POWER9 setup I was using to port other stuff. ZFS worked great for any vdev setup on any OS compiled with the same endianness, but plugging in so much as a thumb drive previously provisioned from a different endian host... insta-console-vomit. Definitely an uncommon problem though :)
    - rincebrain 5y ago
      What? CPU endianness switches work fine, IME - I have pools I originated on SPARC Solaris that work fine on x86, I just got a patch landed for an edge case in one recent feature not interoperating properly between endiannesses but other than that it's worked fine for me. What kind of issue did you see?
      - sennight 5y ago
        > CPU endianness switches work fine Thats great news, hopefully that rare potential problem got addressed in the last year or so. Steps to recreate: compile linux (maybe FreeBSD as well, that is a little more fuzzy) for LE. Boot new kernel and setup a new pool. Recompile for BE. Boot new kernel and enjoy bootloop... lol, actually things are starting to come more into focus... page size played a role in the whole thing. As I said, edge case :) No potential for data loss though.
        rincebrain 5y ago
        I'm guessing this was probably PPC, given the rarity of the other options? Sounds like a fun experiment to run down. Maybe I'll go look when I'm done the current nightmare I'm tinkering with.
        sennight 5y ago
        Yup, Raptor Blackbird. I'd send along my notes if I didn't think they'd be more misleading than helpful. Left that one in a mess that'd be difficult to untangle, but I remember the trail basically ending in a file that was practically unchanged since the original source got released. Because of the code vintage I really wanted to be sure that I didn't introduce performance regressions, so that led to me turning my attention to live kernel debugging over serial, which led to uncovering an even more ancient tty bug (seriously, we're talking unix fountainhead old). She swallowed the spider to catch the fly... I'm presently writing something to crack 8051 firmware xnor'd by a 64 byte key, I don't remember how - but the previously described ZFS edge case somehow got me to this point.
- Volundr 5y ago
  `zpool export` on the machine your taking the array out of and `zpool import <pool name>` on the receiving end will do the job in most cases.
flyinghamster 5y ago
Hopefully, the pool was exported before you moved the disks. I came very close to losing data (but ultimately didn't) when I yanked a pool out of an OpenIndiana system and tried to import it on Ubuntu without exporting it first. ZFS really didn't like finding the disks on devices with radically different names than where it had left off. It ended up re-importing OK when I brought it back to the OpenIndiana box. Whew! I then did an export, and Ubuntu was then able to import the pool.
- boomboomsubban 5y ago
  When you import without exporting, all you risk losing is the last few seconds of data transactions. If a motherboard dying was enough to lose all your data people wouldn't really recommend ZFS. So hopefully the pool was exported, but the worst case scenario isn't very bad.
- flexd 5y ago
  I make my pools with the serial number /dev/disk/by-id/ names, like ata-WDC_WD30EFRX-68EUZN0_WD-XXXXXXXXXXX. That is not going to change between different machines as far as I know, I might be wrong about this. I made some bad choices (in hindsight) when making this pool for my uses, but it's still going strong many years later. [1] This current pool (which is now old, 50k+ power on hours on these disk) has survived a motherboard dying randomly and one or two SSD failures with the OS on. Always just installed a OS on a new SSD and it's been picked up just fine. [1] I went with RAID-Z2 for 6x3TB WD Reds where I probably should have made mirrored pools or something like it to gain more space? It's been a while since I looked at it. Can't really expand this pool or add more storage without replacing each disk one by one with something bigger. I could make another pool with new disks but I'd lose another 20% to parity.
  - flyinghamster 5y ago
    I use those names as well, at least now that I entirely use ZoL. That wasn't an option back in the day with OpenSolaris and Illumos. You'd be using names like c0t0d0 instead, and woe to you if it found the wrong disks in the wrong places (it would just refuse to import claiming the pool was faulted). Caveat: things may have changed since then - it was at least six years ago.

by dkh 5y ago

pdimitar 5y ago

How exactly did you do that? I'm learning ZFS still and I'm planning to move some of the disks in my first NAS to a better machine. Which ZFS commands?

Filligree 5y ago
Nothing special is necessary. All you need to do is import the pool as normal. There’s a potential problem, in that you can’t import a poll on an OS version that lacks the feature flags that are enabled on the pool. The way to solve that is to choose a common subset when creating the pool.
- sennight 5y ago
  Unless they fixed it, another edge case is a change in processor endianness. I started working on a patch a long time ago, but it necessitated changes to code that had remained unchanged since the original release... not a blamelog I'm totally comfortable with.
  - Filligree 5y ago
    I hadn’t heard of that one. Does send |receive still work?
    - sennight 5y ago
      Nope, the system would instantly panic - presumably in order to protect data integrity... imagine the damage possible otherwise. It was actually a fairly ponderous debugging effort, as I remember it, because it was occurring on an already shaky POWER9 setup I was using to port other stuff. ZFS worked great for any vdev setup on any OS compiled with the same endianness, but plugging in so much as a thumb drive previously provisioned from a different endian host... insta-console-vomit. Definitely an uncommon problem though :)
  - rincebrain 5y ago
    What? CPU endianness switches work fine, IME - I have pools I originated on SPARC Solaris that work fine on x86, I just got a patch landed for an edge case in one recent feature not interoperating properly between endiannesses but other than that it's worked fine for me. What kind of issue did you see?
    - sennight 5y ago
      > CPU endianness switches work fine Thats great news, hopefully that rare potential problem got addressed in the last year or so. Steps to recreate: compile linux (maybe FreeBSD as well, that is a little more fuzzy) for LE. Boot new kernel and setup a new pool. Recompile for BE. Boot new kernel and enjoy bootloop... lol, actually things are starting to come more into focus... page size played a role in the whole thing. As I said, edge case :) No potential for data loss though.
      - rincebrain 5y ago
        I'm guessing this was probably PPC, given the rarity of the other options? Sounds like a fun experiment to run down. Maybe I'll go look when I'm done the current nightmare I'm tinkering with.
        sennight 5y ago
        Yup, Raptor Blackbird. I'd send along my notes if I didn't think they'd be more misleading than helpful. Left that one in a mess that'd be difficult to untangle, but I remember the trail basically ending in a file that was practically unchanged since the original source got released. Because of the code vintage I really wanted to be sure that I didn't introduce performance regressions, so that led to me turning my attention to live kernel debugging over serial, which led to uncovering an even more ancient tty bug (seriously, we're talking unix fountainhead old). She swallowed the spider to catch the fly... I'm presently writing something to crack 8051 firmware xnor'd by a 64 byte key, I don't remember how - but the previously described ZFS edge case somehow got me to this point.
Volundr 5y ago
`zpool export` on the machine your taking the array out of and `zpool import <pool name>` on the receiving end will do the job in most cases.

flyinghamster 5y ago

Hopefully, the pool was exported before you moved the disks. I came very close to losing data (but ultimately didn't) when I yanked a pool out of an OpenIndiana system and tried to import it on Ubuntu without exporting it first. ZFS really didn't like finding the disks on devices with radically different names than where it had left off. It ended up re-importing OK when I brought it back to the OpenIndiana box. Whew! I then did an export, and Ubuntu was then able to import the pool.

boomboomsubban 5y ago
When you import without exporting, all you risk losing is the last few seconds of data transactions. If a motherboard dying was enough to lose all your data people wouldn't really recommend ZFS. So hopefully the pool was exported, but the worst case scenario isn't very bad.
flexd 5y ago
I make my pools with the serial number /dev/disk/by-id/ names, like ata-WDC_WD30EFRX-68EUZN0_WD-XXXXXXXXXXX. That is not going to change between different machines as far as I know, I might be wrong about this. I made some bad choices (in hindsight) when making this pool for my uses, but it's still going strong many years later. [1] This current pool (which is now old, 50k+ power on hours on these disk) has survived a motherboard dying randomly and one or two SSD failures with the OS on. Always just installed a OS on a new SSD and it's been picked up just fine. [1] I went with RAID-Z2 for 6x3TB WD Reds where I probably should have made mirrored pools or something like it to gain more space? It's been a while since I looked at it. Can't really expand this pool or add more storage without replacing each disk one by one with something bigger. I could make another pool with new disks but I'd lose another 20% to parity.
- flyinghamster 5y ago
  I use those names as well, at least now that I entirely use ZoL. That wasn't an option back in the day with OpenSolaris and Illumos. You'd be using names like c0t0d0 instead, and woe to you if it found the wrong disks in the wrong places (it would just refuse to import claiming the pool was faulted). Caveat: things may have changed since then - it was at least six years ago.