4 ms·

I'm intending to write up some docs and announce it in the next few days, but since this thread is here, I might as well mention it now: On the topic of altern

I'm intending to write up some docs and announce it in the next few days, but since this thread is here, I might as well mention it now: On the topic of alternative btrfs implementations, I've been working on https://git.lukeshu.com/btrfs-progs-ng/ https://git.lukeshu.com/btrfs-progs-ng/ which is written entirely in Go. Its `btrfs-rec inspect mount --pv=/dev/whatever` is a read-only FUSE implementation of btrfs that is more fault-tolerant (of corrupt filesystems) than the normal in-kernel btrfs driver (and even more tolerant than `btrfs rescue` or `btrfs recover`). It's still missing a bit; RAID almost certainly doesn't work, and encryption is not implemented. But hopefully some folks will find it useful, or at least neat! Oh, comparison with the existing https://github.com/adam900710/btrfs-fuse https://github.com/adam900710/btrfs-fuse : (1) Again, mine has better fault tolerance, (2) but mine is read-only, (3) mine supports xattrs (TODO in Adam's), (4) mine supports separate inode address spaces for subvolumes (Adam's doesn't due to limitations in FUSE, mine works around this by lazily setting up separate mountpoints for each subvolume).

michaelmrose 3y ago
That is really neat have you considered contributing this to btrfs as a recovery option or putting out an ISO with this as a recovery tool?
- LukeShu 3y ago
  I do intend to contribute it to the main btrfs-progs. Like I said, I've got some things to write up in the next few days to try to accomplish that.
Conan_Kudo 3y ago
> On the topic of alternative btrfs implementations, I've been working on https://git.lukeshu.com/btrfs-progs-ng/ https://git.lukeshu.com/btrfs-progs-ng/ which is written entirely in Go. Is this actually a blessed "next-generation" implementation? If not, could you please rename it so it doesn't confuse people?
- tadfisher 3y ago
  If it's better, it should just be named btrfs-progs so folks get confused into installing superior tools.
  - LukeShu 3y ago
    First of all, I do intend for it to get incorporated in to upstream btrfs-progs. Plus, for the most part, it's not a replacement for what's in btrfs-progs, it's additional tools: Replace: - `btrfs-rec inspect dump-trees` is a clone of `btrfs inspect-internal dump-tree`, but it isn't meant to replace it, I only implemented it so that I could diff the output to validate that my implementation agrees with the btrfs-progs implementation. - `btrfs-rec inspect mount` is conceptually a replacement for `btrfs restore`. - `btrfs-rec inspect rebuild-mappings` is a replacement for `btrfs rescue chunk-recover` Additional: - I don't believe that `btrfs-rec inspect rebuild-trees` has an equivalent. - `btrfs-rec inspect list-nodes`, `btrfs-rec inspect ls-files`, and `btrfs-rec inspect ls-trees` have no equivalent in btrfs-progs. - (for completeness) `btrfs-rec inspect spew-items` has no equivalent, but it's for debugging btrfs-rec itself. Missing: - `btrfs-rec` doesn't try to do anything with mounted filesystems, such as `btrfs balance`, `btrfs filesystem`, `btrfs scrub`, `btrfs send`/`btrfs receive`, ... - `btrfs-rec` doesn't try to do anything with filesystem initialization, such as `btrfstune` or `mkfs.btrfs`. - `btrfs-rec` doesn't try to do things with recovering the superblock, I've found `btrfs rescue super-recover` and `btrfs-select-super` to be adequate. - In general, I didn't try to re-implement commands that already work well.
- LukeShu 3y ago
  I intend for it to get incorporated in to btrfs-progs. In the mean-time: yeah, btrfs-progs-ng isn't a great name for the repo, but as I said, I isn't quite ready to publish it yet either, so the repo name wasn't at the top of my list. I originally had the repo named "btrfs-tools", but that's even more confusing (but for different reasons)! (As a side note, I've had others fork software I maintain and call it "-ng", you learn to take it as a compliment :) )
sm_1024 3y ago
I wonder how hard it would be to replace FUSE with userspace block driver. That seems to be focused on performance. https://lwn.net/Articles/903855/ https://lwn.net/Articles/903855/
- kimixa 3y ago
  Isn't that implementing a block device, so the level below the filesystem?
  - LukeShu 3y ago
    Yeah. That said, I've become disillusioned with FUSE over the course of writing this. FUSE is... not great, it's just what we're stuck with. Suggestions as to alternatives to FUSE would be welcome.
LukeShu 3y ago
Too late to edit now: I wrote `btrfs recover`, but there is no such command, I meant to write `btrfs restore`.

michaelmrose 3y ago

That is really neat have you considered contributing this to btrfs as a recovery option or putting out an ISO with this as a recovery tool?

LukeShu 3y ago
I do intend to contribute it to the main btrfs-progs. Like I said, I've got some things to write up in the next few days to try to accomplish that.

Conan_Kudo 3y ago

> On the topic of alternative btrfs implementations, I've been working on https://git.lukeshu.com/btrfs-progs-ng/ https://git.lukeshu.com/btrfs-progs-ng/ which is written entirely in Go. Is this actually a blessed "next-generation" implementation? If not, could you please rename it so it doesn't confuse people?

tadfisher 3y ago
If it's better, it should just be named btrfs-progs so folks get confused into installing superior tools.
- LukeShu 3y ago
  First of all, I do intend for it to get incorporated in to upstream btrfs-progs. Plus, for the most part, it's not a replacement for what's in btrfs-progs, it's additional tools: Replace: - `btrfs-rec inspect dump-trees` is a clone of `btrfs inspect-internal dump-tree`, but it isn't meant to replace it, I only implemented it so that I could diff the output to validate that my implementation agrees with the btrfs-progs implementation. - `btrfs-rec inspect mount` is conceptually a replacement for `btrfs restore`. - `btrfs-rec inspect rebuild-mappings` is a replacement for `btrfs rescue chunk-recover` Additional: - I don't believe that `btrfs-rec inspect rebuild-trees` has an equivalent. - `btrfs-rec inspect list-nodes`, `btrfs-rec inspect ls-files`, and `btrfs-rec inspect ls-trees` have no equivalent in btrfs-progs. - (for completeness) `btrfs-rec inspect spew-items` has no equivalent, but it's for debugging btrfs-rec itself. Missing: - `btrfs-rec` doesn't try to do anything with mounted filesystems, such as `btrfs balance`, `btrfs filesystem`, `btrfs scrub`, `btrfs send`/`btrfs receive`, ... - `btrfs-rec` doesn't try to do anything with filesystem initialization, such as `btrfstune` or `mkfs.btrfs`. - `btrfs-rec` doesn't try to do things with recovering the superblock, I've found `btrfs rescue super-recover` and `btrfs-select-super` to be adequate. - In general, I didn't try to re-implement commands that already work well.
LukeShu 3y ago
I intend for it to get incorporated in to btrfs-progs. In the mean-time: yeah, btrfs-progs-ng isn't a great name for the repo, but as I said, I isn't quite ready to publish it yet either, so the repo name wasn't at the top of my list. I originally had the repo named "btrfs-tools", but that's even more confusing (but for different reasons)! (As a side note, I've had others fork software I maintain and call it "-ng", you learn to take it as a compliment :) )

sm_1024 3y ago

I wonder how hard it would be to replace FUSE with userspace block driver. That seems to be focused on performance. https://lwn.net/Articles/903855/ https://lwn.net/Articles/903855/

kimixa 3y ago
Isn't that implementing a block device, so the level below the filesystem?
- LukeShu 3y ago
  Yeah. That said, I've become disillusioned with FUSE over the course of writing this. FUSE is... not great, it's just what we're stuck with. Suggestions as to alternatives to FUSE would be welcome.

LukeShu 3y ago

Too late to edit now: I wrote `btrfs recover`, but there is no such command, I meant to write `btrfs restore`.