4 ms·

bcachefs is a really interesting FS in the linux space. Checkout some of the writing about it here: https://www.patreon.com/bcachefs https://www.patreon.com/bc

bcachefs is a really interesting FS in the linux space. Checkout some of the writing about it here: https://www.patreon.com/bcachefs https://www.patreon.com/bcachefs

ansible 8y ago
Well, that is interesting. I'm curious that they're talking about implementing encryption at the filesystem layer. This has typically been done elsewhere. And anyway, key management is an issue. Seems like it would be beyond the scope for a filesystem. But I had similar thoughts right before I started learning about the write anywhere layout used by NetApp all those years ago, where mixing layers had enormous benefits (ZFS too). The idea of incorporating a flash translation layer (FTL) is interesting, but there is the hardware support issue. Meaning that yes, you can still buy raw NAND flash memory, but what are you going to connect it to? NAND flash controllers which can present a useful interface to the host processor (like USB) already incorporate a FTL. Similarly, eMMC memory has a FTL layer baked in, and presents itself as a block-addressable device. Raw NAND flash controllers are increasingly rare on modern microcontrollers.
- cyphar 8y ago
  ZFS encryption works in a similar fashion, and ext4 also has encryption (which is used by default in Android). Personally I'm not a huge fan of this because unlike the clear benefits of giving filesystems control over raw devices, full disk encryption has requirements that can't really be provided partially. You want to ensure no metadata about the filesystem structure or files will be known without knowing the encryption key but this is clearly not possible without having block device layer encryption. ZFS encryption allows for all sorts of useful operations on encrypted filesystems without having the key. This is cool, but also obviously provides a lot of information about the filesystem structure. Also dedup tables aren't encrypted in this setup. ext4 only encrypts filenames and contents, which is worse in some respects.
  - exikyut 8y ago
    > Also dedup tables aren't encrypted in this setup. This sounds like low-hanging fruit for a security researcher to try their hand at: comprehend the dedupe table format, and see what you can derive from it (basically the real-world version of https://xkcd.com/704/ https://xkcd.com/704/).
  - koverstreet 8y ago
    The only metadata stored in the clear in bcachefs is the superblock and the very first part of the btree node and journal entry headers - just the checksum, magic number (identifying it as a btree node/journal header), and a flags field which mainly contains the checksum/encryption type. So an attacker can tell roughly how much metadata you have, but literally nothing about the contents.
- exikyut 8y ago
  > the write anywhere layout used by NetApp all those years ago I've never heard of "write anywhere" before but it sounds interesting. What is/was this? > ... incorporating a flash translation layer (FTL) is interesting, but ... what are you going to connect it to? NAND flash controllers ... already incorporate a FTL. ... Raw NAND flash controllers are increasingly rare on modern microcontrollers. My read of the FTL bit is that of hopeful optimization that supports the ideal-case - say, the filesystem equivalent of all the revving-up everyone is doing to support RISC-V. It's clearly leaning pretty heavily on the bit about existing precedent. My practical hope is that the code stays in good, well-tended condition up to the point it can be usefully taken advantage of - and that someone notices it to good effect sooner rather than later.
  - justinsaccount 8y ago
    > I've never heard of "write anywhere" before but it sounds interesting. What is/was this? https://en.wikipedia.org/wiki/Write_Anywhere_File_Layout https://en.wikipedia.org/wiki/Write_Anywhere_File_Layout
- RX14 8y ago
  A really good reason to use encryption at the filesystem level is that you can use stream ciphers instead of block ciphers (because you have a place to store nonces). The current disk encryption is based on XTS, which has a section in wikipedia on it's weaknesses [1]. [1] https://en.wikipedia.org/wiki/Disk_encryption_theory#XTS https://en.wikipedia.org/wiki/Disk_encryption_theory#XTS
zaarn 8y ago
I'm very excited for bcachefs, I've been trying it out on my personal laptop for a while (until I was forced to reinstall). It was a very pleasant experience and I hope that it gets mainlined in the close future. Linux desperately needs ZFS with better integration and more flexibility.
- Valmar 8y ago
  Why were you forced to reinstall? Was it due to bcachefs in some way? Is there anything you felt was still missing back when you were using it?
  - zaarn 8y ago
    It wasn't bcachefs' fault (in fact, I'm copying my laptop's install back on a bcachefs partition), it was a accumulative problem where in the end the bcachefs-enabled kernel and various tools decided to go on strike and forced me to go back on a btrfs based partition. (I booted into a post-mount environment and ran rsync)
nabla9 8y ago
First: > zfs is block based, not extent based, whereas all other modern filesystems have been extent based for years: the reason they did this is that extents plus snapshots are really hard. then: >Snapshots: In progress. bcachefs's snapshot implementation is going to be significantly more capable than any competing implementations. When it's done, ... Multiple ZFS challengers can get easily to the point where they have 80-90% of the cool features that ZFS has. Then everything slows down to crawl because getting the same performance and reliability + features is super hard. When you leave the essential but hard features 'for later' they are harder to add later without big rewrite (or losing performance). Ticking all the boxes with good performance becomes exponentially harder as number of features grow. There comes time to order things: sacrifice performance in this to attain this.
- RX14 8y ago
  Kent's a really capable and experienced guy, he has a design in mind and I have every confidence he'll get snapshots working and fast. But even if that's not true, bcachefs still has an excellent feature set already with great performance, especially in long-tail latency.
  - nabla9 8y ago
    It's not about being excellent or not, or if he can implement something that technically works. He or any other excellent guy don't have an idea how the implementation he designs works in multiple workloads and different machines and disk drives. There will be time period of several years of adjusting and tinkering and compromising.
- Hello71 8y ago
  bcachefs is specifically trying to avoid the btrfs garbage fire by building slowly on top of well-tested bcache. I am confident it won't end up like btrfs where RAID5/6 is still horribly broken yet people claim the filesystem is totally stable and fine for production use.
  - std_throwawayay 8y ago

bcachefs is a really interesting FS in the linux space. Checkout some of the writing about it here: https://www.patreon.com/bcachefs https://www.patreon.com/bcachefs

ansible 8y ago

Well, that is interesting. I'm curious that they're talking about implementing encryption at the filesystem layer. This has typically been done elsewhere. And anyway, key management is an issue. Seems like it would be beyond the scope for a filesystem. But I had similar thoughts right before I started learning about the write anywhere layout used by NetApp all those years ago, where mixing layers had enormous benefits (ZFS too). The idea of incorporating a flash translation layer (FTL) is interesting, but there is the hardware support issue. Meaning that yes, you can still buy raw NAND flash memory, but what are you going to connect it to? NAND flash controllers which can present a useful interface to the host processor (like USB) already incorporate a FTL. Similarly, eMMC memory has a FTL layer baked in, and presents itself as a block-addressable device. Raw NAND flash controllers are increasingly rare on modern microcontrollers.

cyphar 8y ago
ZFS encryption works in a similar fashion, and ext4 also has encryption (which is used by default in Android). Personally I'm not a huge fan of this because unlike the clear benefits of giving filesystems control over raw devices, full disk encryption has requirements that can't really be provided partially. You want to ensure no metadata about the filesystem structure or files will be known without knowing the encryption key but this is clearly not possible without having block device layer encryption. ZFS encryption allows for all sorts of useful operations on encrypted filesystems without having the key. This is cool, but also obviously provides a lot of information about the filesystem structure. Also dedup tables aren't encrypted in this setup. ext4 only encrypts filenames and contents, which is worse in some respects.
- exikyut 8y ago
  > Also dedup tables aren't encrypted in this setup. This sounds like low-hanging fruit for a security researcher to try their hand at: comprehend the dedupe table format, and see what you can derive from it (basically the real-world version of https://xkcd.com/704/ https://xkcd.com/704/).
- koverstreet 8y ago
  The only metadata stored in the clear in bcachefs is the superblock and the very first part of the btree node and journal entry headers - just the checksum, magic number (identifying it as a btree node/journal header), and a flags field which mainly contains the checksum/encryption type. So an attacker can tell roughly how much metadata you have, but literally nothing about the contents.
exikyut 8y ago
> the write anywhere layout used by NetApp all those years ago I've never heard of "write anywhere" before but it sounds interesting. What is/was this? > ... incorporating a flash translation layer (FTL) is interesting, but ... what are you going to connect it to? NAND flash controllers ... already incorporate a FTL. ... Raw NAND flash controllers are increasingly rare on modern microcontrollers. My read of the FTL bit is that of hopeful optimization that supports the ideal-case - say, the filesystem equivalent of all the revving-up everyone is doing to support RISC-V. It's clearly leaning pretty heavily on the bit about existing precedent. My practical hope is that the code stays in good, well-tended condition up to the point it can be usefully taken advantage of - and that someone notices it to good effect sooner rather than later.
- justinsaccount 8y ago
  > I've never heard of "write anywhere" before but it sounds interesting. What is/was this? https://en.wikipedia.org/wiki/Write_Anywhere_File_Layout https://en.wikipedia.org/wiki/Write_Anywhere_File_Layout
RX14 8y ago
A really good reason to use encryption at the filesystem level is that you can use stream ciphers instead of block ciphers (because you have a place to store nonces). The current disk encryption is based on XTS, which has a section in wikipedia on it's weaknesses [1]. [1] https://en.wikipedia.org/wiki/Disk_encryption_theory#XTS https://en.wikipedia.org/wiki/Disk_encryption_theory#XTS

zaarn 8y ago

I'm very excited for bcachefs, I've been trying it out on my personal laptop for a while (until I was forced to reinstall). It was a very pleasant experience and I hope that it gets mainlined in the close future. Linux desperately needs ZFS with better integration and more flexibility.

Valmar 8y ago
Why were you forced to reinstall? Was it due to bcachefs in some way? Is there anything you felt was still missing back when you were using it?
- zaarn 8y ago
  It wasn't bcachefs' fault (in fact, I'm copying my laptop's install back on a bcachefs partition), it was a accumulative problem where in the end the bcachefs-enabled kernel and various tools decided to go on strike and forced me to go back on a btrfs based partition. (I booted into a post-mount environment and ran rsync)

nabla9 8y ago

First: > zfs is block based, not extent based, whereas all other modern filesystems have been extent based for years: the reason they did this is that extents plus snapshots are really hard. then: >Snapshots: In progress. bcachefs's snapshot implementation is going to be significantly more capable than any competing implementations. When it's done, ... Multiple ZFS challengers can get easily to the point where they have 80-90% of the cool features that ZFS has. Then everything slows down to crawl because getting the same performance and reliability + features is super hard. When you leave the essential but hard features 'for later' they are harder to add later without big rewrite (or losing performance). Ticking all the boxes with good performance becomes exponentially harder as number of features grow. There comes time to order things: sacrifice performance in this to attain this.

RX14 8y ago
Kent's a really capable and experienced guy, he has a design in mind and I have every confidence he'll get snapshots working and fast. But even if that's not true, bcachefs still has an excellent feature set already with great performance, especially in long-tail latency.
- nabla9 8y ago
  It's not about being excellent or not, or if he can implement something that technically works. He or any other excellent guy don't have an idea how the implementation he designs works in multiple workloads and different machines and disk drives. There will be time period of several years of adjusting and tinkering and compromising.
Hello71 8y ago
bcachefs is specifically trying to avoid the btrfs garbage fire by building slowly on top of well-tested bcache. I am confident it won't end up like btrfs where RAID5/6 is still horribly broken yet people claim the filesystem is totally stable and fine for production use.
- std_throwawayay 8y ago