7 ms·

RaiserFS is dead as far as I've read. btrfs is a shitshow - sorry to having to resort to such words but I will refrain from using it ever again if I have a cho

• giis 11y ago
  Interesting ReiserFS comment http://blog.pgaddict.com/posts/friends-dont-let-friends-use-btrfs-for-oltp#comment-2266022309 http://blog.pgaddict.com/posts/friends-dont-let-friends-use-...
• simoncion 11y ago
  > * RAID is utterly broken: - http://www.spinics.net/lists/linux-btrfs/msg48561.html http://www.spinics.net/lists/linux-btrfs/msg48561.html sigh From the message that you linked: "FWIW... Older btrfs userspace such as your v3.17 is "OK" for normal runtime use, assuming you don't need any newer features, as in normal runtime, it's the kernel code doing the real work and userspace for the most part simply makes the appropriate kernel calls to do that work. But, once you get into a recovery situation like the one you're in now, current userspace becomes much more mportant, as the various things you'll do to attempt recovery rely far more on userspace code directly accessing the filesystem, and it's only the newest userspace code that has the latest fixes." If you look, you see that the guy reporting the problem was using a very new kernel, but a one-year-old btrfs-tools package. It makes no sense to use tools that are substantially older than the kernel you're running. He might have had much better luck with recovery (that is, he might have been able to get a rw volume with all his data, rather than an ro volume -with all his data- out of the process) with a recent tools package. > http://www.spinics.net/lists/linux-btrfs/msg47845.html http://www.spinics.net/lists/linux-btrfs/msg47845.html Maybe I'm reading this one wrong, but it looks like the summary of the issues are: * For multi-disk arrays with disks of varying sizes, each smaller disk does not receive data until all larger disks have as much or less free space than the smaller disk in question. [0] * The read scheduling algo is currently naive, using PID evenness to determine the disk in a pair to read, rather than current IO load on the disk. Did I miss something, or misunderstand something? If I didn't miss anything, I don't see how you can get "RAID is utterly broken" from "RAID has less-than-optimal write and read scheduling". > * It's useless for databases or VM images (disabling datacow is a ugly hack IMHO) I can't agree. I use btrfs on spinning rust to store my multi-terabyte Postgres 9.4 DB. It works well enough for my low-to-medium-volume [1] workload. > * Lot's of ENOSPC bugs (just look at the mailing list) I don't run into those, and haven't for literal years. > * Quota is utterly broken... Yes, quota is a difficult problem that continues to be unsolved. > * You need to run the latest kernel... Yes. Agreed. > ...if not: Data corruption bugs ... I haven't run into any, ever on the handful of systems on which I use BTRFS. > ...if not: ... deadlocks True. There was that unlink/move deadlock in 4.1. I hit it from time to time when updating my copy of the metadata from the Gentoo Portage tree. Happily, it only blocked operation on the affected file. The remainder of the system kept running along, and a -ugh- reboot unblocked the operation. [2] Look. I'm not saying that btrfs is feature-complete, suitable for every workload, or that any given person should use it for their workload. I'm just saying that the situation is far less dire than you're making it out to be. I've been using it as the rootfs for at least one of my daily use systems for the past 5.5 years, and haven't had any trouble out of it [3] in the past 2+ years. [0] That phrasing was abysmal. Here's an example: Given an array of one each of 3GB, 2GB, 1.5GB devices. the 2GB device will get data after 1GB has been written to the 3GB device. The 1.5GB device will get data after 0.5GB of additional data has been written to each of the 3GB and 2GB devices (making a total of 1GB additional data written to the array before the 1.5GB device starts getting data). AIUI, if you're running in RAID1 mode, then the same logic applies, but with paired devices. [1] Most definitely not web-scale workload. [2] I don't know if a unmount/remount cycle would have also worked. The wedged operation happened on my rootfs, so I had no way to test this. [3] Except for the unlink/move deadlock in 4.1.
  • nisa 11y ago
    Hi, sorry for my tone - it's beyond rational discussion - however I still think it's not totally coming from nowhere: [RAID] >sigh From the message that you linked: The part below your quote is interesting. > General note about btrfs and btrfs raid. Given that btrfs itself remains a "stabilizing, but not yet fully mature and stable filesystem", while btrfs raid will often let you recover from a bad device, sometimes that recovery is in the form of letting you mount ro, so you can access the data and copy it elsewhere, before blowing away the filesystem and starting over. I have hotplug and online-replace with ZFS and mdraid. btrfs wants a special degraded remount that might or might not be able to recover to rw someday... replacing a disk online is not really possible. Handling a failing drive is not really implemented. Don't call it stable or don't call it RAID. It's nice that I'm able to pull of the data of my ro-mounted degraded array when a disk fails.. however all of this stuff is not really documented and was very surprising to me. If you have production machine e.g. on btrfs RAID10 it sucks to not be able to replace a broken drive on the fly online. ZFS/mdraid just do it.. [RAID II] > * The read scheduling algo is currently naive, using PID evenness to determine the disk in a pair to read, rather than current IO load on the disk. > Did I miss something, or misunderstand something? If I didn't miss anything, I don't see how you can get "RAID is utterly broken" from "RAID has less-than-optimal write and read scheduling". Walking is a less than than optimal way to move from Moscow to Berlin. If I take a train I expect a train - btrfs is a train that occasionally stops and says: Well, just walk... okay, you see I'm kind of emotional invested in this shit. However it's not what's usually expected as RAID and it's not documented. We have here a lot of applications that do exactly that: a single PID reads lots of stuff very often sequentially... so RAID1 doubles throughput. Well not on btrfs. [ENOSPC] > I don't run into those, and haven't for literal years. 4x4TB RAID10 (90% free) -> ENOSPC - this was somehwere between 3.16 or 3.19 - how can this even happen? [Corruption] > I haven't run into any, ever on the handful of systems on which I use BTRFS. Good for you. Was hit by this nasty undeletable directory bug: http://www.spinics.net/lists/linux-btrfs/msg35789.html http://www.spinics.net/lists/linux-btrfs/msg35789.html And then this: http://www.spinics.net/lists/linux-btrfs/msg45108.html http://www.spinics.net/lists/linux-btrfs/msg45108.html Umounting and running btrfs --check repair is not really cool if you want some uptime... [Deadlocks] Currently btrfs deadlocks reliable in a OOM situation... With 4.2 things got better, before running some medium heavy Hadoop job or ElastichSearch indexing killed 20% to 50% of the machines due to deadlocks (with 3.19 IIRC) > Look. I'm not saying that btrfs is feature-complete, suitable for every workload, or that any given person should use it for their workload. I'm just saying that the situation is far less dire than you're making it out to be. I've been using it as the rootfs for at least one of my daily use systems for the past 5.5 years, and haven't had any trouble out of it [3] in the past 2+ years. It's probably fine for personal use. But don't think about using it for something that is demanding. You can google the presentations that say: Use it! It's stable! Enterprise! It's not. Maybe in 5 years. I just saw it breaking in too many strange ways to bother anymore with it.