4 ms·

Is btrfs a good choice for a format for a regular Linux install on a dev or user machine now? I've been using ext4 for almost a decade, curious to hear response

by leshow 8y ago

Is btrfs a good choice for a format for a regular Linux install on a dev or user machine now? I've been using ext4 for almost a decade, curious to hear responses.

stingraycharles 8y ago
For the past few months, I've been trying to make it work (mostly because of the raid + snapshotting functionality), but I have ran into nothing but trouble. Frequent hangs of my system (unrelated to the defrag process that runs daily, which also uses 100% CPU for minutes -- the best I have deduced it towards is either because of using raid+nvme, or because of high (>100GB) RAM)), Docker is very unstable using btrfs [2], etc etc. I moved back to ext4 + mdraid just last week and couldn't be happier. 1 https://serverfault.com/questions/747366/btrfs-write-operations-hang-when-appending-to-files https://serverfault.com/questions/747366/btrfs-write-operati... 2 https://github.com/moby/moby/issues/34501 https://github.com/moby/moby/issues/34501
- jplayer01 8y ago
  Seems like these are issues mainly related to using older versions of the kernel and/or Docker.
  - stingraycharles 8y ago
    This is not the case. I've done a lot of trial & error, and tried Ubuntu 18.04 / 18.10, Mint and Arch. The problems persist everywhere. Best to my knowledge, it started when I upgraded from 32GB RAM to 128GB RAM, and/or started using virtual machines more intensively after that. I tried tuning btrfs in various ways, disabling CoW, enabling/disabling autodefrag. The problems start to re-appear and intensify after about one week of using a fresh install. All I know it's definitely not something simple like "use the latest kernel".
- nicolaslem 8y ago
  Every year I consider spending some time to get familiar with btrfs and every year I read horror stories like this one and I stop right there.
  - ahartmetz 8y ago
    Just look at the btrfs commits in each new Linux version - the critical fixes never stop. And most of them are direct corner case fixes without systematic cleanup. The code seems to consist of corner cases without a robust framework to perform difficult operations.
- shmerl 8y ago
  Do you have a recent kernel? Filesystems get fixes only with kernel updates.
- cmurf 8y ago
  I've been using it for years, including with Docker, and haven't had any problems. There are people successfully using Btrfs with tens of thousands of containers, e.g. https://lore.kernel.org/linux-btrfs/CAMp4zn8YUdVShFibUKCXtwZTZpicCbmm7zSYMn7+K5CNt-cxGA@mail.gmail.com/ https://lore.kernel.org/linux-btrfs/CAMp4zn8YUdVShFibUKCXtwZ... Why are you using defrag daily? Why aren't you using the autodefrag mount option instead if you really need such frequent defragging? Really anytime there's a problem in the kernel, you need to try the workload with a mainline kernel and report the problem to the upstream kernel list if you can reproduce it. If you can't reproduce the problem with mainline, then you have to take up the bug with your distro. That's the way it is with everything, not just Btrfs. Ergo, I think asking for technical help about Btrfs on serverfault or stack exchange or even HN is weird. People having Btrfs problems need to go to directly to the upstream list: http://vger.kernel.org/vger-lists.html#linux-btrfs http://vger.kernel.org/vger-lists.html#linux-btrfs Even weirder is the serverfault user has SLES! He has a support contract with SUSE so why post in serverfault? It just makes zero sense to me to do that... And the github link, the OP was asked for more information as it sounded like not a Docker problem at all, and no followup response.
  - stingraycharles 8y ago
    I've been using Arch Linux mostly, with the 4.18 and more recently 4.19 kernels. As explained in another post, I've done extensive trial & error to isolate the issue. I understand that it's just a single data point, and other people have more positive experiences. My intention was not to find a solution here on HN, but OP asked for experience reports so I gave mine.
    - cmurf 8y ago
      Using Arch, and such recent kernels, and having isolated the issue, you should report it upstream. Otherwise it sounds like you have the time and preference to report negative experiences on HN rather than see the problem get fixed.
  - evil-olive 8y ago
    > Really anytime there's a problem in the kernel, you need to try the workload with a mainline kernel and report the problem to the upstream kernel list if you can reproduce it. If you can't reproduce the problem with mainline, then you have to take up the bug with your distro. That's the way it is with everything, not just Btrfs. Sure, if I have a bug with program X that's patched by distro Y, that's the normal support path. I think there's a couple issues with applying that same logic to a filesystem that's supposedly stable enough to be used as a root FS though. First, a filesystem should generally be stable enough that patches applied by a distro don't completely ruin it. How many bugs have there been in ext4 or xfs that were specific to one distro and their kernel patches? They're certainly possible, but I would think that pre-release testing would catch the vast majority of them. Red Hat dropping support of btrfs was a big vote of no confidence here, because it's not just lack of support for RHEL users, it implies lack of testing efforts even for Centos & Fedora users. Second, if I'm having issues with my root filesystem, it's a bit of a crapshoot as to whether the system is stable enough to compile a mainline kernel and try to reproduce the bug while running that. And finally, I simply don't want bugs in my root filesystem. I don't even want to get to the point of pondering if I should send the bug report to my distro or to the mainline kernel. I want my root filesystem to be a thing that Just Works, without having to think about it.
dx87 8y ago
It depends on how you use it, I tried it a couple of years ago and ran into a couple of issues with the tooling around it. I used it on a fresh install of OpenSuse and ran into issues a little while later because the distro was saving btrfs snapshots every day, and they eventually filled up my small SSD and prevented the system from being able to boot because it didn't have enough free space to create some small temporary files it needed. You'll also need to get used to using btrfs tools certain tasks, because tools like df and du don't understand how much space btrfs is actually using so you get innacurate results. Things may have changed in the past couple of years, but having to tune different paramaters and learn new commands wasn't really worth the effort because I wasn't using most of the features on a standard Linux desktop install.
- leshow 8y ago
  I wouldn't be using any of the advanced features. I just want a filesystem. The automatic compression stuff seemed attractive, but if there's anything else I have to do besides just format a drive differently it's not worth it for me. Thanks for sharing your experience.
shmerl 8y ago
I'm using btrfs on home computer (with compression enabled) on big storage HDD (games, videos, photos etc.), primarily because of compression and easiness of creating and mounting subvolumes without making new partitions. For regular root / home partition on NVMe I'm using XFS which is faster.
- leshow 8y ago
  How much compression do you get out of video files? My understanding is they are already pretty compressed so you don't see a great improvement. I have many terabytes of files on some drives, even 5% of savings could go a long way.
  - shmerl 8y ago
    I didn't play with forced compression, so most of such files aren't compressed by btrfs. With forced compression, overall ratio will be higher. But if you have raw video, I expect it to kick in more. I.e. for my archive volume compression is quite low (99%). For example for games volume it's better (81%).
corndoge 8y ago
I use it in raid 1 on a 2x1TB setup and it works great, if you don't use its 'exotic' features like software raid and the fancy volume management stuff just stick to ext4
- leshow 8y ago
  fair enough, I don't need any of the fancy features.
gh02t 8y ago
I've been using BTRFS for at least 5 years on my primary workstation and various other machines and have had no issues whatsoever. Being able to format the root of a drive with BTRFS and then work with subvolumes instead of partitions and manipulate them on the fly is wonderful, as are filesystem snapshots. No more planning partition layouts and/or dealing with LVM. I would (and do) use ZFS on a server because it's rock solid and powerful, but I prefer BTRFS on desktops / laptops because it's more flexible and lower maintenance. The only major downside for me is that it pretty much only works on Linux.
- DelightOne 8y ago
  Do you use raid with BTRFS?
  - gh02t 8y ago
    I have. I didn't have any trouble out of it.
- ofrzeta 8y ago
  There's also WinBtrfs https://github.com/maharmstone/btrfs https://github.com/maharmstone/btrfs
  - Leace 8y ago
    Judging by the issues on WinBtrfs it doesn't work well (or at all).
  - gh02t 8y ago
    Oh, I didn't know that existed. I'm not sure if I'd trust it, but glad to know it exists. It's not really a big issue for me, though, as I have a home NAS. Really more important to me than Windows support is support on BSD (esp. FreeBSD) as I use that on my NAS.
rurban 8y ago
I'm using it for root for years. Stable, but slow. Still much faster than NTFS or Apple hfs+ though. zfs needs to much diskspace for my taste.
- leshow 8y ago
  I would never use ntfs or apple's filesystem for linux though.
ofrzeta 8y ago
Also take a look at https://btrfs.wiki.kernel.org/index.php/Status https://btrfs.wiki.kernel.org/index.php/Status

by leshow 8y ago

Is btrfs a good choice for a format for a regular Linux install on a dev or user machine now? I've been using ext4 for almost a decade, curious to hear responses.

stingraycharles 8y ago

For the past few months, I've been trying to make it work (mostly because of the raid + snapshotting functionality), but I have ran into nothing but trouble. Frequent hangs of my system (unrelated to the defrag process that runs daily, which also uses 100% CPU for minutes -- the best I have deduced it towards is either because of using raid+nvme, or because of high (>100GB) RAM)), Docker is very unstable using btrfs [2], etc etc. I moved back to ext4 + mdraid just last week and couldn't be happier. 1 https://serverfault.com/questions/747366/btrfs-write-operations-hang-when-appending-to-files https://serverfault.com/questions/747366/btrfs-write-operati... 2 https://github.com/moby/moby/issues/34501 https://github.com/moby/moby/issues/34501

jplayer01 8y ago
Seems like these are issues mainly related to using older versions of the kernel and/or Docker.
- stingraycharles 8y ago
  This is not the case. I've done a lot of trial & error, and tried Ubuntu 18.04 / 18.10, Mint and Arch. The problems persist everywhere. Best to my knowledge, it started when I upgraded from 32GB RAM to 128GB RAM, and/or started using virtual machines more intensively after that. I tried tuning btrfs in various ways, disabling CoW, enabling/disabling autodefrag. The problems start to re-appear and intensify after about one week of using a fresh install. All I know it's definitely not something simple like "use the latest kernel".
nicolaslem 8y ago
Every year I consider spending some time to get familiar with btrfs and every year I read horror stories like this one and I stop right there.
- ahartmetz 8y ago
  Just look at the btrfs commits in each new Linux version - the critical fixes never stop. And most of them are direct corner case fixes without systematic cleanup. The code seems to consist of corner cases without a robust framework to perform difficult operations.
shmerl 8y ago
Do you have a recent kernel? Filesystems get fixes only with kernel updates.
cmurf 8y ago
I've been using it for years, including with Docker, and haven't had any problems. There are people successfully using Btrfs with tens of thousands of containers, e.g. https://lore.kernel.org/linux-btrfs/CAMp4zn8YUdVShFibUKCXtwZTZpicCbmm7zSYMn7+K5CNt-cxGA@mail.gmail.com/ https://lore.kernel.org/linux-btrfs/CAMp4zn8YUdVShFibUKCXtwZ... Why are you using defrag daily? Why aren't you using the autodefrag mount option instead if you really need such frequent defragging? Really anytime there's a problem in the kernel, you need to try the workload with a mainline kernel and report the problem to the upstream kernel list if you can reproduce it. If you can't reproduce the problem with mainline, then you have to take up the bug with your distro. That's the way it is with everything, not just Btrfs. Ergo, I think asking for technical help about Btrfs on serverfault or stack exchange or even HN is weird. People having Btrfs problems need to go to directly to the upstream list: http://vger.kernel.org/vger-lists.html#linux-btrfs http://vger.kernel.org/vger-lists.html#linux-btrfs Even weirder is the serverfault user has SLES! He has a support contract with SUSE so why post in serverfault? It just makes zero sense to me to do that... And the github link, the OP was asked for more information as it sounded like not a Docker problem at all, and no followup response.
- stingraycharles 8y ago
  I've been using Arch Linux mostly, with the 4.18 and more recently 4.19 kernels. As explained in another post, I've done extensive trial & error to isolate the issue. I understand that it's just a single data point, and other people have more positive experiences. My intention was not to find a solution here on HN, but OP asked for experience reports so I gave mine.
  - cmurf 8y ago
    Using Arch, and such recent kernels, and having isolated the issue, you should report it upstream. Otherwise it sounds like you have the time and preference to report negative experiences on HN rather than see the problem get fixed.
- evil-olive 8y ago
  > Really anytime there's a problem in the kernel, you need to try the workload with a mainline kernel and report the problem to the upstream kernel list if you can reproduce it. If you can't reproduce the problem with mainline, then you have to take up the bug with your distro. That's the way it is with everything, not just Btrfs. Sure, if I have a bug with program X that's patched by distro Y, that's the normal support path. I think there's a couple issues with applying that same logic to a filesystem that's supposedly stable enough to be used as a root FS though. First, a filesystem should generally be stable enough that patches applied by a distro don't completely ruin it. How many bugs have there been in ext4 or xfs that were specific to one distro and their kernel patches? They're certainly possible, but I would think that pre-release testing would catch the vast majority of them. Red Hat dropping support of btrfs was a big vote of no confidence here, because it's not just lack of support for RHEL users, it implies lack of testing efforts even for Centos & Fedora users. Second, if I'm having issues with my root filesystem, it's a bit of a crapshoot as to whether the system is stable enough to compile a mainline kernel and try to reproduce the bug while running that. And finally, I simply don't want bugs in my root filesystem. I don't even want to get to the point of pondering if I should send the bug report to my distro or to the mainline kernel. I want my root filesystem to be a thing that Just Works, without having to think about it.

dx87 8y ago

It depends on how you use it, I tried it a couple of years ago and ran into a couple of issues with the tooling around it. I used it on a fresh install of OpenSuse and ran into issues a little while later because the distro was saving btrfs snapshots every day, and they eventually filled up my small SSD and prevented the system from being able to boot because it didn't have enough free space to create some small temporary files it needed. You'll also need to get used to using btrfs tools certain tasks, because tools like df and du don't understand how much space btrfs is actually using so you get innacurate results. Things may have changed in the past couple of years, but having to tune different paramaters and learn new commands wasn't really worth the effort because I wasn't using most of the features on a standard Linux desktop install.

leshow 8y ago
I wouldn't be using any of the advanced features. I just want a filesystem. The automatic compression stuff seemed attractive, but if there's anything else I have to do besides just format a drive differently it's not worth it for me. Thanks for sharing your experience.

shmerl 8y ago

I'm using btrfs on home computer (with compression enabled) on big storage HDD (games, videos, photos etc.), primarily because of compression and easiness of creating and mounting subvolumes without making new partitions. For regular root / home partition on NVMe I'm using XFS which is faster.

leshow 8y ago
How much compression do you get out of video files? My understanding is they are already pretty compressed so you don't see a great improvement. I have many terabytes of files on some drives, even 5% of savings could go a long way.
- shmerl 8y ago
  I didn't play with forced compression, so most of such files aren't compressed by btrfs. With forced compression, overall ratio will be higher. But if you have raw video, I expect it to kick in more. I.e. for my archive volume compression is quite low (99%). For example for games volume it's better (81%).

corndoge 8y ago

I use it in raid 1 on a 2x1TB setup and it works great, if you don't use its 'exotic' features like software raid and the fancy volume management stuff just stick to ext4

leshow 8y ago
fair enough, I don't need any of the fancy features.

gh02t 8y ago

I've been using BTRFS for at least 5 years on my primary workstation and various other machines and have had no issues whatsoever. Being able to format the root of a drive with BTRFS and then work with subvolumes instead of partitions and manipulate them on the fly is wonderful, as are filesystem snapshots. No more planning partition layouts and/or dealing with LVM. I would (and do) use ZFS on a server because it's rock solid and powerful, but I prefer BTRFS on desktops / laptops because it's more flexible and lower maintenance. The only major downside for me is that it pretty much only works on Linux.

DelightOne 8y ago
Do you use raid with BTRFS?
- gh02t 8y ago
  I have. I didn't have any trouble out of it.
ofrzeta 8y ago
There's also WinBtrfs https://github.com/maharmstone/btrfs https://github.com/maharmstone/btrfs
- Leace 8y ago
  Judging by the issues on WinBtrfs it doesn't work well (or at all).
- gh02t 8y ago
  Oh, I didn't know that existed. I'm not sure if I'd trust it, but glad to know it exists. It's not really a big issue for me, though, as I have a home NAS. Really more important to me than Windows support is support on BSD (esp. FreeBSD) as I use that on my NAS.

rurban 8y ago

I'm using it for root for years. Stable, but slow. Still much faster than NTFS or Apple hfs+ though. zfs needs to much diskspace for my taste.

leshow 8y ago
I would never use ntfs or apple's filesystem for linux though.

ofrzeta 8y ago

Also take a look at https://btrfs.wiki.kernel.org/index.php/Status https://btrfs.wiki.kernel.org/index.php/Status