4 ms·

I know this is going to sound trite, but this is precisely the reason I don't use mdadm I really don't like lvm either, but in some cases its the only real way

I know this is going to sound trite, but this is precisely the reason I don't use mdadm I really don't like lvm either, but in some cases its the only real way to join larger hardware arrays together (for example some arrays will not let you stripe multiple raid7s instead insisting that you have a 50drive raid 7 array. No dice.) This is where ZFS really does shine here. The control, visibility and documentation are just wonderful. Its almost as good as reading a redbook. For the hobbyist I don;'t see any reason why you'd really want to have an mdadm array, barring the slight edge of performance when you are running at >75% capacity.

louwrentius 6y ago
I think this has nothing to do with MD vs ZFS. The key issue is that I think it's crazy to tinker around with 'bad blocks' in the first place. A hard drive with bad blocks that it cannot conceal from the end-user is broken and not to be trusted. Replace it.
zrm 6y ago
You should see what some of the other alternatives do. I just decommissioned a server and pulled the drives out of it. One of them had SMART data indicating that it reached the "drive failing, replace now" stage six months prior. The hardware RAID controller in the server had been content to give no indication of this at all. Had another one where the drives were in two external containers and one of them lost power for a moment. The hardware RAID controller thereby marked that half of the drives as failed. This was too many failures to run the array, but it also had no mechanism to assemble an array using the perfectly operational drives it had marked as failed, so that was a restore from backup. Meanwhile mdadm now supports --write-journal which moots the only real advantage hardware RAID ever held.
pessimizer 6y ago
The reason you don't use mdadm is because it keeps track of bad blocks by default, although that can be disabled, but when using the debian installer to set up arrays it sets up the config file in /tmp then blows away any changes made when dropping out of the installer to modify the temp conf on return to the installer, making it impossible to set up a md array that doesn't track bad blocks when using a serial console (although if you aren't, you can open up another terminal to do it) for the install? That's precisely the reason you don't use mdadm?
McGlockenshire 6y ago
> For the hobbyist I don;'t see any reason why you'd really want to have an mdadm array The "don't even dare to use ZFS without dedicating 8-16 GB of memory to it" warnings all over the ZFS documentation online has served as a very, very good reason for me to not use ZFS for anything at all. I'll stick with tooling that lets my homelab actually have enough RAM to run the applications I want to run.
- Sebb767 6y ago
  > don't even dare to use ZFS without dedicating 8-16 GB of ECC [!] memory to it Fixed that for you. Now, I've since build a storage server for my homelab using ZFS, but mdadm is far more accessible. It also feels easier - I know what's on the drives and how it is stored. In case of something strange happening with ZFS, I don't have any hope except backups. Also, ZFS lacks support in most distributions and somewhat obscure encryption capability. I don't mean to bash ZFS here - it's great. But the GP feels like it's saying "I don't see why would anyone not want to run a dedicated kubernetes cluster with HA for running a program on Linux".
  - KaiserPro 6y ago
    > GP feels like it's saying "I don't see why would anyone not want to run a dedicated kubernetes cluster with HA for running a program on Linux". Thats not quite right Its more that the documentation, command line interface and general user setup is far more friendly than mdadm. I've been a sysadmin for many years, and looked after innumerable file servers, from clustered XFS (!) through many years of lustre, plain old XFS/ext2/3/5 on HWraid to netapps&isilons, and all sorts of flash in the pan appliances. Barring GPFS, the only file system that has documentation and tooling that is simple to understand and doesn't immediately bite you in the arse is ZFS. Kubernetes is neither simple, easy or quick to setup, or run.
- KaiserPro 6y ago
  that warning is the same for any File server. if you want performance, then you need ram. otherwise you don't have enough space to keep hot blocks. ZFS works as well as any FS in places with 8 gigs of ram. Hell I even run a KVM node with just 16 gigs of ram. Its working just fine.
Scramblejams 6y ago
Because as a hobbyist, the fact that I can expand my RAID6 array one drive at a time is, for me, mdadm's killer app.

louwrentius 6y ago

I think this has nothing to do with MD vs ZFS. The key issue is that I think it's crazy to tinker around with 'bad blocks' in the first place. A hard drive with bad blocks that it cannot conceal from the end-user is broken and not to be trusted. Replace it.

zrm 6y ago

You should see what some of the other alternatives do. I just decommissioned a server and pulled the drives out of it. One of them had SMART data indicating that it reached the "drive failing, replace now" stage six months prior. The hardware RAID controller in the server had been content to give no indication of this at all. Had another one where the drives were in two external containers and one of them lost power for a moment. The hardware RAID controller thereby marked that half of the drives as failed. This was too many failures to run the array, but it also had no mechanism to assemble an array using the perfectly operational drives it had marked as failed, so that was a restore from backup. Meanwhile mdadm now supports --write-journal which moots the only real advantage hardware RAID ever held.

pessimizer 6y ago

The reason you don't use mdadm is because it keeps track of bad blocks by default, although that can be disabled, but when using the debian installer to set up arrays it sets up the config file in /tmp then blows away any changes made when dropping out of the installer to modify the temp conf on return to the installer, making it impossible to set up a md array that doesn't track bad blocks when using a serial console (although if you aren't, you can open up another terminal to do it) for the install? That's precisely the reason you don't use mdadm?

McGlockenshire 6y ago

> For the hobbyist I don;'t see any reason why you'd really want to have an mdadm array The "don't even dare to use ZFS without dedicating 8-16 GB of memory to it" warnings all over the ZFS documentation online has served as a very, very good reason for me to not use ZFS for anything at all. I'll stick with tooling that lets my homelab actually have enough RAM to run the applications I want to run.

Sebb767 6y ago
> don't even dare to use ZFS without dedicating 8-16 GB of ECC [!] memory to it Fixed that for you. Now, I've since build a storage server for my homelab using ZFS, but mdadm is far more accessible. It also feels easier - I know what's on the drives and how it is stored. In case of something strange happening with ZFS, I don't have any hope except backups. Also, ZFS lacks support in most distributions and somewhat obscure encryption capability. I don't mean to bash ZFS here - it's great. But the GP feels like it's saying "I don't see why would anyone not want to run a dedicated kubernetes cluster with HA for running a program on Linux".
- KaiserPro 6y ago
  > GP feels like it's saying "I don't see why would anyone not want to run a dedicated kubernetes cluster with HA for running a program on Linux". Thats not quite right Its more that the documentation, command line interface and general user setup is far more friendly than mdadm. I've been a sysadmin for many years, and looked after innumerable file servers, from clustered XFS (!) through many years of lustre, plain old XFS/ext2/3/5 on HWraid to netapps&isilons, and all sorts of flash in the pan appliances. Barring GPFS, the only file system that has documentation and tooling that is simple to understand and doesn't immediately bite you in the arse is ZFS. Kubernetes is neither simple, easy or quick to setup, or run.
KaiserPro 6y ago
that warning is the same for any File server. if you want performance, then you need ram. otherwise you don't have enough space to keep hot blocks. ZFS works as well as any FS in places with 8 gigs of ram. Hell I even run a KVM node with just 16 gigs of ram. Its working just fine.

Scramblejams 6y ago

Because as a hobbyist, the fact that I can expand my RAID6 array one drive at a time is, for me, mdadm's killer app.