4 ms·

Over the recent years on every new laptop install i switched between filesystems, so i had ext3/4, btrfs and (currently) xfs on my system. I have to say, btrfs

by buster 9y ago

Over the recent years on every new laptop install i switched between filesystems, so i had ext3/4, btrfs and (currently) xfs on my system. I have to say, btrfs had the most glitches (a few years back), although it worked ok'ish (no data loss). Nowadays, i must say that i very much prefer a stable filesystem with as little complicated logic as possible. I actually never use snapshots or subtrees! I never put another disk in my laptop (where would that go?!) so i don't need to do dynamic resizing (while online of course!). All this makes the filesystems a lot more complex then it has to be. I've also run into problems using ZFS on Solaris some years back which took ~2 weeks in dtracing what the hell is going on. Of course it was related to CoW. My lessons learned: Check your requirements. Will you really need and use subtrees/snapshots/XYZ on your system? Will you really need to do online-resizing? If not, just use a stable, simple filesystem. There are perfect usecases for ZFS or btrfs. But not everyone needs the advanced features.

viraptor 9y ago
> Will you really need and use subtrees/snapshots/XYZ on your system? It's a valid question, but not the best one. Almost nobody needs snapshots. But they make things easier. You most likely don't need a journaled fs in your laptop either (battery level notification should take care of the issues). But it does make life better. "Need" is not the threshold I'm interested in. Most features, I'd like. One feature I think I do need most is scrubbing, which is still absent from most filesystems :(
- buster 9y ago
  "Need" as in "will you use it?". I played around with snapshots once and never really used them. So i clearly don't have a need for them on my laptop. Journaling on the other hand helps data safety a lot and i think it's not overly complex. I've had data loss happing in the past before journaling, but never again since then. So, wouldn't i need CoW for even better "data safety"? Maybe, but since i've never experienced data loss for so many years, i don't feel like the added complexity is worth it. On my laptop, for my usecase. But that's only me. Your experience may differ very much :)
  - lomnakkus 9y ago
    > "Need" as in "will you use it?". Well, many of us have experienced a botched system package upgrade or two. If the file system supports snapshots, then the package manager could automatically ensure fully atomic package upgrades. That should be reason enough, I should think. Re: The data loss issue: Yes, I've actually have XFS completely throw away a file system upon a hard power-off + boot-up cycle. (This was ages ago, I'm sure it's improved heaps since then.)
    - cyphar 9y ago
      > If the file system supports snapshots, then the package manager could automatically ensure fully atomic package upgrades. That's exactly what openSUSE / SLE do with snapper. Every upgrade or package install with YaST/zypper creates two snapshots (before/after) and you can easily rollback to an older snapshot (even doing so from GRUB). This has been enabled by default for years.
    - buster 9y ago
      Good point. I'm using Debian as my Desktop for many years. I don't remember a "botched" system package upgrade in the last 5 years, but i've probably learned over years how to handle dpkg/apt. The atomic updating is a very interesting topic and the reason why i find ostree/guix/nixos very appealing. Note that neither ostree nor guix or nixos make use of filesystem snapshots, afaik. OSTree even documents why it won't use filesystem snapshots: https://ostree.readthedocs.io/en/latest/manual/related-projects/#combining-dpkgrpm-btrfslvm https://ostree.readthedocs.io/en/latest/manual/related-proje... Debians dpkg does not use snapshots as well. So, it's a definitely a nice-to-have, but not something i need, because i can handle dpkg/apt much better then i could handle filesystem internals. That's sort of the point: I don't want the complexity in the filesystem, but i am fine with it in userspace. I can use snapshots on filesystem level. Or i can use other backup tools in userspace. While it's certainly neat that the filesystem can do that, i'm perfectly fine with handling backups on another level. Another example: It's certainly neat that there are a bunch of distributed filesystems (which by the way have A LOT of complexity and often can't handle all workloads you would expect from a filesystem). But i'd rather use either an S3-like network storage or build a system that scales well without relying on Ceph/Gluster/Quobyte/etc. For example, in a hypothetical distributed system i'd rather use Cassandra and distribute data over commodity hardware then use Ceph. I'd rather handle problems with data persistence/replication on the cassandra level then debugging on file system level. Especially, when Cassandra has a problem i'll most likely be able to access all data atleast on the filesystem level. When my filesystem is borked, i'm in a much worse situation.
- exikyut 9y ago
  If you're on an SSD/MTD/NVMe, you have TRIM, and scrubbing is a no-op no matter what approach you try. You need a spinning HDD for scrubbing to be useful. Here is one way to do simple, secure scrubbing on Linux without any intrusive system changes. It is mildly restrictive, but works. First, you need a small, dedicated partition, but it only needs to be around 16MB or so. Resizing an existing partition down (tune2fs will happily resize a mounted ext4 filesystem, but you'll probably still need to reboot to reload the partition table once you've resized that too) will give you a bit of space. Now you have a small area of the disk that occupies a known range of sectors, and because you have no TRIM, writes to this area will be properly deterministic. Good. Create and mount a new filesystem without a journal on the new partition. ext2 could work here (:D), you could `mkfs.ext{3,4} -O ^has_journal`, or you could use filesystem defaults and simply overwrite the entire partition with /dev/urandom later. Make a sparse file with fallocate (make sure the file system you create the file on can handle sparse files) that is big enough to handle the biggest file. Create a LUKS volume with a detached header inside the new sparse file, and store the detached header metadata into a file in the new journal-less partition. Create an ordinary filesystem inside the LUKS volume. Now you have a Rube Goldberg sparse file. You've moved the deterministic-writing/journal-less stage into a tiny key, which is a lot easier to manage than a whole gigabytes+-large partition. As an alternative you could drop the key onto a flash drive, and nuke the flash drive when you wanted to kill the data. That's kind of wasteful though (and it carries the same flash-drive-quality risks as copying the only copy of the data itself onto the flash drive). LUKS was designed such that if you lose the key(s) or the detached header, all that's left is statistically random garbage.
  - dsr_ 9y ago
    You seem to be using a different definition of scrubbing than people talking about filesystems usually use. Scrubbing means to read all the data off a filesystem and compare it against its checksums, so that you are confident nothing has happened to the data (hardware failures, cosmic rays, whatever). ZFS and btrfs have specific scrub commands that do that. There's no scrubbing available for a system which does not keep some form of checksum/crc/hash of the data. I think that you are talking about secure delete procedures.
eternauta3k 9y ago
It's ok to play with the OS in order to learn.
lorenzfx 9y ago
I'm using ZFS on my FreeBSD laptop. Snapshots not only make backups safer (by making sure the complete backup is taken at the same time, and by zfs sending and receiving them), the boot environments feature also make upgrading safer. I also really like that I don't need to partition my disk, if it turns out that /tmp needs > 10% of the disk for whatever reason: no problem! And as I like my data, I appreciate checksumming and copy-on-write. I haven't noticed any bad slowdowns compared to ext4 on my Debian laptop I used before.
- rsync 9y ago
  "I'm using ZFS on my FreeBSD laptop. Snapshots not only make backups safer..." Did you know you can 'zfs send' snapshots to rsync.net ?[1][2] [1] https://arstechnica.com/information-technology/2015/12/rsync-net-zfs-replication-to-the-cloud-is-finally-here-and-its-fast/ https://arstechnica.com/information-technology/2015/12/rsync... [2] http://www.rsync.net/products/zfsintro.html http://www.rsync.net/products/zfsintro.html
Maakuth 9y ago
It's a good practice to use LVM there between disk partitions and volumes. It has negligible performance implications but makes things very flexible when you need to resize volumes or add space. You also gain reliable snapshotting from device mapper, although that does have some performance effects.
pmlnr 9y ago
My requirements: transparent compression. I'm left with: btrfs or, if possible, ZFS.

by buster 9y ago

viraptor 9y ago

> Will you really need and use subtrees/snapshots/XYZ on your system? It's a valid question, but not the best one. Almost nobody needs snapshots. But they make things easier. You most likely don't need a journaled fs in your laptop either (battery level notification should take care of the issues). But it does make life better. "Need" is not the threshold I'm interested in. Most features, I'd like. One feature I think I do need most is scrubbing, which is still absent from most filesystems :(

buster 9y ago
"Need" as in "will you use it?". I played around with snapshots once and never really used them. So i clearly don't have a need for them on my laptop. Journaling on the other hand helps data safety a lot and i think it's not overly complex. I've had data loss happing in the past before journaling, but never again since then. So, wouldn't i need CoW for even better "data safety"? Maybe, but since i've never experienced data loss for so many years, i don't feel like the added complexity is worth it. On my laptop, for my usecase. But that's only me. Your experience may differ very much :)
- lomnakkus 9y ago
  > "Need" as in "will you use it?". Well, many of us have experienced a botched system package upgrade or two. If the file system supports snapshots, then the package manager could automatically ensure fully atomic package upgrades. That should be reason enough, I should think. Re: The data loss issue: Yes, I've actually have XFS completely throw away a file system upon a hard power-off + boot-up cycle. (This was ages ago, I'm sure it's improved heaps since then.)
  - cyphar 9y ago
    > If the file system supports snapshots, then the package manager could automatically ensure fully atomic package upgrades. That's exactly what openSUSE / SLE do with snapper. Every upgrade or package install with YaST/zypper creates two snapshots (before/after) and you can easily rollback to an older snapshot (even doing so from GRUB). This has been enabled by default for years.
  - buster 9y ago
    Good point. I'm using Debian as my Desktop for many years. I don't remember a "botched" system package upgrade in the last 5 years, but i've probably learned over years how to handle dpkg/apt. The atomic updating is a very interesting topic and the reason why i find ostree/guix/nixos very appealing. Note that neither ostree nor guix or nixos make use of filesystem snapshots, afaik. OSTree even documents why it won't use filesystem snapshots: https://ostree.readthedocs.io/en/latest/manual/related-projects/#combining-dpkgrpm-btrfslvm https://ostree.readthedocs.io/en/latest/manual/related-proje... Debians dpkg does not use snapshots as well. So, it's a definitely a nice-to-have, but not something i need, because i can handle dpkg/apt much better then i could handle filesystem internals. That's sort of the point: I don't want the complexity in the filesystem, but i am fine with it in userspace. I can use snapshots on filesystem level. Or i can use other backup tools in userspace. While it's certainly neat that the filesystem can do that, i'm perfectly fine with handling backups on another level. Another example: It's certainly neat that there are a bunch of distributed filesystems (which by the way have A LOT of complexity and often can't handle all workloads you would expect from a filesystem). But i'd rather use either an S3-like network storage or build a system that scales well without relying on Ceph/Gluster/Quobyte/etc. For example, in a hypothetical distributed system i'd rather use Cassandra and distribute data over commodity hardware then use Ceph. I'd rather handle problems with data persistence/replication on the cassandra level then debugging on file system level. Especially, when Cassandra has a problem i'll most likely be able to access all data atleast on the filesystem level. When my filesystem is borked, i'm in a much worse situation.
exikyut 9y ago
If you're on an SSD/MTD/NVMe, you have TRIM, and scrubbing is a no-op no matter what approach you try. You need a spinning HDD for scrubbing to be useful. Here is one way to do simple, secure scrubbing on Linux without any intrusive system changes. It is mildly restrictive, but works. First, you need a small, dedicated partition, but it only needs to be around 16MB or so. Resizing an existing partition down (tune2fs will happily resize a mounted ext4 filesystem, but you'll probably still need to reboot to reload the partition table once you've resized that too) will give you a bit of space. Now you have a small area of the disk that occupies a known range of sectors, and because you have no TRIM, writes to this area will be properly deterministic. Good. Create and mount a new filesystem without a journal on the new partition. ext2 could work here (:D), you could `mkfs.ext{3,4} -O ^has_journal`, or you could use filesystem defaults and simply overwrite the entire partition with /dev/urandom later. Make a sparse file with fallocate (make sure the file system you create the file on can handle sparse files) that is big enough to handle the biggest file. Create a LUKS volume with a detached header inside the new sparse file, and store the detached header metadata into a file in the new journal-less partition. Create an ordinary filesystem inside the LUKS volume. Now you have a Rube Goldberg sparse file. You've moved the deterministic-writing/journal-less stage into a tiny key, which is a lot easier to manage than a whole gigabytes+-large partition. As an alternative you could drop the key onto a flash drive, and nuke the flash drive when you wanted to kill the data. That's kind of wasteful though (and it carries the same flash-drive-quality risks as copying the only copy of the data itself onto the flash drive). LUKS was designed such that if you lose the key(s) or the detached header, all that's left is statistically random garbage.
- dsr_ 9y ago
  You seem to be using a different definition of scrubbing than people talking about filesystems usually use. Scrubbing means to read all the data off a filesystem and compare it against its checksums, so that you are confident nothing has happened to the data (hardware failures, cosmic rays, whatever). ZFS and btrfs have specific scrub commands that do that. There's no scrubbing available for a system which does not keep some form of checksum/crc/hash of the data. I think that you are talking about secure delete procedures.

eternauta3k 9y ago

It's ok to play with the OS in order to learn.

lorenzfx 9y ago

I'm using ZFS on my FreeBSD laptop. Snapshots not only make backups safer (by making sure the complete backup is taken at the same time, and by zfs sending and receiving them), the boot environments feature also make upgrading safer. I also really like that I don't need to partition my disk, if it turns out that /tmp needs > 10% of the disk for whatever reason: no problem! And as I like my data, I appreciate checksumming and copy-on-write. I haven't noticed any bad slowdowns compared to ext4 on my Debian laptop I used before.

rsync 9y ago
"I'm using ZFS on my FreeBSD laptop. Snapshots not only make backups safer..." Did you know you can 'zfs send' snapshots to rsync.net ?[1][2] [1] https://arstechnica.com/information-technology/2015/12/rsync-net-zfs-replication-to-the-cloud-is-finally-here-and-its-fast/ https://arstechnica.com/information-technology/2015/12/rsync... [2] http://www.rsync.net/products/zfsintro.html http://www.rsync.net/products/zfsintro.html

Maakuth 9y ago

It's a good practice to use LVM there between disk partitions and volumes. It has negligible performance implications but makes things very flexible when you need to resize volumes or add space. You also gain reliable snapshotting from device mapper, although that does have some performance effects.

pmlnr 9y ago

My requirements: transparent compression. I'm left with: btrfs or, if possible, ZFS.