7 ms·

One btrfs bug which is 100% reproducible: * Start with an ext3 filesystem 70% full. * Convert to btrfs using btrfs-convert. * Delete the ext3_saved snapshot

One btrfs bug which is 100% reproducible: * Start with an ext3 filesystem 70% full. * Convert to btrfs using btrfs-convert. * Delete the ext3_saved snapshot of the original filesystem as recommended by the convert utility. * Enable compression (-o compress) and defrag the filesystem as recommended by the man page for how to compress all existing data. It fails with out of disk space, leaving a filesystem which isn't repairable - deleting files will not free any space. The fact such a bug seems to have existed for years, with such a basic following of the man pages for a common use case (migration to btrfs to make use of its compression abilities to get more free space), tells me that it isn't yet ready for primetime.

cesarb 3y ago
> deleting files will not free any space. Does a rebalance fix it? I have once (and only once, back when it was new) hit a "out of disk space" situation with btrfs, and IIRC rebalancing was enough to fix it. > for a common use case It might have been a common use case back when btrfs was new (though I doubt it, most users of btrfs probably created the filesystem from scratch even back then), but I doubt it's a common use case nowadays.
- eru 3y ago
  > It might have been a common use case back when btrfs was new (though I doubt it, most users of btrfs probably created the filesystem from scratch even back then), but I doubt it's a common use case nowadays. It's perhaps not as common as it once was, but you'd expect it to be common enough to work, and not some obscure corner case.
- bayindirh 3y ago
  From my perspective, A filesystem is a critical infrastructure in an OS, and failing here and there and not fixing these bugs because they're not common is not acceptable. Same for the RAID5/6 bugs in BTRFS. What's their solution? A simple warning in the docs: > RAID5/6 has known problems and should not be used in production. [0] Also the CLI discourages you from creating these things. Brilliant. This is why I don't use BTRFS anywhere. An FS shall be bulletproof. Errors must only cause from hardware problems. Not random bugs in a filesystem. [0]: https://btrfs.readthedocs.io/en/latest/mkfs.btrfs.html#multiple-devices https://btrfs.readthedocs.io/en/latest/mkfs.btrfs.html#multi...
  - chronid 3y ago
    Machines die. Hardware has bugs, or is broken. Things just bork. It's a fact of life. Would I build a file storage system around btrfs? No - without proper redundancy at least. But I'm told at least Synology does. I'm pretty sure there's plenty of cases where it's perfectly usable - the feature set it has today is plenty useful and the worst case scenario is an host reimage. I can live with that. applications will generally break production ten billion times before btrfs does.
    - bayindirh 3y ago
      > Machines dies. Hardware has bugs, or is broken. Things just bork. It's a fact of life. I know, I'm a sysadmin. I care for hardware, mend it, heal it, and sometimes donate, cann-bird or bury it. I'm used to it. > worst case scenario is an host reimage... While hosting PBs of data on it? No, thanks. > Would I build a file storage system around btrfs? No - without proper redundancy at least. Everything is easy for small n. When you store 20TB on 4x5TB drives, everything can be done. When you have a >5PB of storage on racks, you need at least a copy of that system running hot-standby. That's not cheap in any sense. Instead, I'd use ZFS, Lustre, anything, but not BTRFS. > I can live with that - applications will generally break production ten billion times before btrfs does. In our case, no. Our systems doesn't stop because a daemon decided to stop because a server among many fried itself.
      - chronid 3y ago
        I have worked on and around systems with an order of magnitude more data and a single node failing did not matter. We weren't using btrfs anyway (for data drives) and it definitely was not cheap. But storage never is. But again, most systems are not like that. Kubernetes cluster nodes? Reimage at will. Compute nodes for vms backed by SAN? Reimage at will. Btrfs can actually make that reimage faster and it's pretty reliable on a single flash drive so why not?
        bayindirh 3y ago
        Well, that was my primary point. BTRFS is not ready for these kind of big installations handled by ZFS or Lustre at this point. On the other hand, BTRFS’ single disk performance, esp, for small files is visibly lower than EXT4 and XFS, so why bother? There are many solutions for EXT4 which allows versioning, and if I can reimage a node (or 200) in 5 minute flat, why should I bother with the overhead of BTRFS? It’s not that I haven’t tried BTRFS. Its features are nice, but from my perspective, it’s not ready for prime time, yet. What bothers me is the mental gymnastics pretending that it’s mature at this point. It’ll be good file system. An excellent one in fact, but it still needs to cook.
- lxgr 3y ago
  There’s literally no way I could migrate my NAS other than through an in-place FS conversion since it’s >>50% full. The same probably applies to may consumer devices.
- chasil 3y ago
  A rebalance means that every file on the filesystem will be rewritten. This is drastic, and I'd rather perform such an operation on an image copy. This is one case where ZFS is absolutely superior; if a drive goes offline, and is returned to a set at a later date, the resilver only touches the changed/needed blocks. Btrfs forces the entire filesystem to be rewritten in a rebalance, which is much more drastic. I am very willing to allow ZFS mirrors to be degraded; I would never, ever let this happen to btrfs if at all avoidable.
  - o11c 3y ago
    The desired "compress every file" operation will also cause every file on the filesystem to be rewritten though ...
orev 3y ago
As a Linux user since kernel 0.96, I have never once considered doing an in-place migration to a new file system. That seems like a crazy thing to try to do, and I would hope it’s only done as a last resort and with all data fully backed up before trying it. I would agree that if this is presented in the documentation as something it supports, then it should work as expected. If it doesn’t work, then a pull request to remove it from the docs might be the best course of action.
- tuyiown 3y ago
  I don't know if you talk only about linux or you meant your comment as a generalization, but have you heard of in place APFS migration from HFS+ ?
  - bayindirh 3y ago
    Similarly Microsoft offered FAT32 to NTFS in-place migration, and it did the required checks before starting to ensure it completes successfully. It was more than 20 years ago IIRC.
    - tuyiown 3y ago
      The notable thing with APFS is that the migration came automatically with an OS update (to high sierra) by default.
      - bayindirh 3y ago
        Yes, I think also mounting ext3 systems as ext4 and migrating it to ext4 as you access it is a good way too. IIRC you had to enable a couple of flags in the ext3 system with tune2fs to enable ext4 features as well, but it didn't break, and migrated as you write & delete files.
- londons_explore 3y ago
  The design of the convert utility is pretty good - the convert is effectively atomic - at any point during conversion, if you kill power midway, the disk is either a valid ext4 filesystem, or a valid btrfs filesystem.
jcalvinowens 3y ago
It's open source: if things don't get fixed, it's because no user cares enough to fix it. I'm certainty not wasting my time on that, nobody uses ext3 anymore! You are as empowered to fix this as anybody else, if it presents a real problem for you.
- bayindirh 3y ago
  You can reliably change ext3 with ext4 in the GP's comment. It's not an extX problem, it's an BTRFS problem. If nobody cares that much, maybe deprecate the tool, then? Also, just because it's Open Source (TM) doesn't mean developers will accept any patch regardless of its quality. Like everything, FOSS is 85% people, 15% code. > You are as empowered to fix this as anybody else, if it presents a real problem for you. I have reported many bugs in Open Source software. If I had the time to study code and author a fix, I'd submit the patch itself, which I was able to do, a couple of times.
  - jcalvinowens 3y ago
    > If nobody cares that much, maybe deprecate the tool, then? If you think that's the right thing to do, you're as free as anybody else to send documentation patches. I doubt anybody would argue with you here, but who knows :) > Also, just because it's Open Source (TM) doesn't mean developers will accept any patch regardless of its quality. Of course not. If you want to make a difference, you have to put in the work. It's worth it IMHO.
    - bayindirh 3y ago
      > If you think that's the right thing to do, you're as free as anybody else to send documentation patches :) That won't do anything. Instead I can start a small commotion by sending a small request (to the mailing lists) to deprecate the tool, which I don't want to do. Because I'm busy. :) Also, I don't like commotions, and prefer civilized discussions. > Of course not. If you want to make a difference, you have to put in the work. It's worth it IMHO. Of course. This is what I do. For example, I have a one liner in Debian Installer. I had a big patch in GDM, but after coordinating with the developers, they decided to not merge the fix + new feature, for example.

cesarb 3y ago

> deleting files will not free any space. Does a rebalance fix it? I have once (and only once, back when it was new) hit a "out of disk space" situation with btrfs, and IIRC rebalancing was enough to fix it. > for a common use case It might have been a common use case back when btrfs was new (though I doubt it, most users of btrfs probably created the filesystem from scratch even back then), but I doubt it's a common use case nowadays.

eru 3y ago
> It might have been a common use case back when btrfs was new (though I doubt it, most users of btrfs probably created the filesystem from scratch even back then), but I doubt it's a common use case nowadays. It's perhaps not as common as it once was, but you'd expect it to be common enough to work, and not some obscure corner case.
bayindirh 3y ago
From my perspective, A filesystem is a critical infrastructure in an OS, and failing here and there and not fixing these bugs because they're not common is not acceptable. Same for the RAID5/6 bugs in BTRFS. What's their solution? A simple warning in the docs: > RAID5/6 has known problems and should not be used in production. [0] Also the CLI discourages you from creating these things. Brilliant. This is why I don't use BTRFS anywhere. An FS shall be bulletproof. Errors must only cause from hardware problems. Not random bugs in a filesystem. [0]: https://btrfs.readthedocs.io/en/latest/mkfs.btrfs.html#multiple-devices https://btrfs.readthedocs.io/en/latest/mkfs.btrfs.html#multi...
- chronid 3y ago
  Machines die. Hardware has bugs, or is broken. Things just bork. It's a fact of life. Would I build a file storage system around btrfs? No - without proper redundancy at least. But I'm told at least Synology does. I'm pretty sure there's plenty of cases where it's perfectly usable - the feature set it has today is plenty useful and the worst case scenario is an host reimage. I can live with that. applications will generally break production ten billion times before btrfs does.
  - bayindirh 3y ago
    > Machines dies. Hardware has bugs, or is broken. Things just bork. It's a fact of life. I know, I'm a sysadmin. I care for hardware, mend it, heal it, and sometimes donate, cann-bird or bury it. I'm used to it. > worst case scenario is an host reimage... While hosting PBs of data on it? No, thanks. > Would I build a file storage system around btrfs? No - without proper redundancy at least. Everything is easy for small n. When you store 20TB on 4x5TB drives, everything can be done. When you have a >5PB of storage on racks, you need at least a copy of that system running hot-standby. That's not cheap in any sense. Instead, I'd use ZFS, Lustre, anything, but not BTRFS. > I can live with that - applications will generally break production ten billion times before btrfs does. In our case, no. Our systems doesn't stop because a daemon decided to stop because a server among many fried itself.
    - chronid 3y ago
      I have worked on and around systems with an order of magnitude more data and a single node failing did not matter. We weren't using btrfs anyway (for data drives) and it definitely was not cheap. But storage never is. But again, most systems are not like that. Kubernetes cluster nodes? Reimage at will. Compute nodes for vms backed by SAN? Reimage at will. Btrfs can actually make that reimage faster and it's pretty reliable on a single flash drive so why not?
      - bayindirh 3y ago
        Well, that was my primary point. BTRFS is not ready for these kind of big installations handled by ZFS or Lustre at this point. On the other hand, BTRFS’ single disk performance, esp, for small files is visibly lower than EXT4 and XFS, so why bother? There are many solutions for EXT4 which allows versioning, and if I can reimage a node (or 200) in 5 minute flat, why should I bother with the overhead of BTRFS? It’s not that I haven’t tried BTRFS. Its features are nice, but from my perspective, it’s not ready for prime time, yet. What bothers me is the mental gymnastics pretending that it’s mature at this point. It’ll be good file system. An excellent one in fact, but it still needs to cook.
lxgr 3y ago
There’s literally no way I could migrate my NAS other than through an in-place FS conversion since it’s >>50% full. The same probably applies to may consumer devices.
chasil 3y ago
A rebalance means that every file on the filesystem will be rewritten. This is drastic, and I'd rather perform such an operation on an image copy. This is one case where ZFS is absolutely superior; if a drive goes offline, and is returned to a set at a later date, the resilver only touches the changed/needed blocks. Btrfs forces the entire filesystem to be rewritten in a rebalance, which is much more drastic. I am very willing to allow ZFS mirrors to be degraded; I would never, ever let this happen to btrfs if at all avoidable.
- o11c 3y ago
  The desired "compress every file" operation will also cause every file on the filesystem to be rewritten though ...

orev 3y ago

As a Linux user since kernel 0.96, I have never once considered doing an in-place migration to a new file system. That seems like a crazy thing to try to do, and I would hope it’s only done as a last resort and with all data fully backed up before trying it. I would agree that if this is presented in the documentation as something it supports, then it should work as expected. If it doesn’t work, then a pull request to remove it from the docs might be the best course of action.

tuyiown 3y ago
I don't know if you talk only about linux or you meant your comment as a generalization, but have you heard of in place APFS migration from HFS+ ?
- bayindirh 3y ago
  Similarly Microsoft offered FAT32 to NTFS in-place migration, and it did the required checks before starting to ensure it completes successfully. It was more than 20 years ago IIRC.
  - tuyiown 3y ago
    The notable thing with APFS is that the migration came automatically with an OS update (to high sierra) by default.
    - bayindirh 3y ago
      Yes, I think also mounting ext3 systems as ext4 and migrating it to ext4 as you access it is a good way too. IIRC you had to enable a couple of flags in the ext3 system with tune2fs to enable ext4 features as well, but it didn't break, and migrated as you write & delete files.
londons_explore 3y ago
The design of the convert utility is pretty good - the convert is effectively atomic - at any point during conversion, if you kill power midway, the disk is either a valid ext4 filesystem, or a valid btrfs filesystem.

jcalvinowens 3y ago

It's open source: if things don't get fixed, it's because no user cares enough to fix it. I'm certainty not wasting my time on that, nobody uses ext3 anymore! You are as empowered to fix this as anybody else, if it presents a real problem for you.

bayindirh 3y ago
You can reliably change ext3 with ext4 in the GP's comment. It's not an extX problem, it's an BTRFS problem. If nobody cares that much, maybe deprecate the tool, then? Also, just because it's Open Source (TM) doesn't mean developers will accept any patch regardless of its quality. Like everything, FOSS is 85% people, 15% code. > You are as empowered to fix this as anybody else, if it presents a real problem for you. I have reported many bugs in Open Source software. If I had the time to study code and author a fix, I'd submit the patch itself, which I was able to do, a couple of times.
- jcalvinowens 3y ago
  > If nobody cares that much, maybe deprecate the tool, then? If you think that's the right thing to do, you're as free as anybody else to send documentation patches. I doubt anybody would argue with you here, but who knows :) > Also, just because it's Open Source (TM) doesn't mean developers will accept any patch regardless of its quality. Of course not. If you want to make a difference, you have to put in the work. It's worth it IMHO.
  - bayindirh 3y ago
    > If you think that's the right thing to do, you're as free as anybody else to send documentation patches :) That won't do anything. Instead I can start a small commotion by sending a small request (to the mailing lists) to deprecate the tool, which I don't want to do. Because I'm busy. :) Also, I don't like commotions, and prefer civilized discussions. > Of course not. If you want to make a difference, you have to put in the work. It's worth it IMHO. Of course. This is what I do. For example, I have a one liner in Debian Installer. I had a big patch in GDM, but after coordinating with the developers, they decided to not merge the fix + new feature, for example.