9 ms·

WinBtrfs – an open-source btrfs driver for Windows

311 points by frankjr 3y ago

090rf 3y ago
[dead]
rkagerer 3y ago
Awesome! Anyone using this long term or in production who can comment on how it's been working? I see TRIM is supported. Is RETRIM also (or whatever is needed during drive optimization to release areas that didn't get TRIMmed the first time due to a full command queue). Could this serve as an effective NTFS replacement with data parity for those who don't like ReFS? How mature is it compared to ZFS on Windows?
- deleted 3y ago
  [deleted]
- jiggawatts 3y ago
  ReFS with Storage Spaces already serves this purpose and is integrated and fully supported. From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong.[1] Using something that is unreliable at best on its native OS shoehorned into Windows is madness. Fun for a lark, sure, but I would never ever entrust this combination with any actual data. [1] “It works for me on my two disk mirror” is an anecdote, not data.
  - rkagerer 3y ago
    Thanks. I tried ReFS when it first came out and it was terribly slow (with data parity on), and Storage Spaces was obscure to set up and manage. Has the landscape improved?
    - mgerdts 3y ago
      On WS2022 without patches I noticed that Storage Spaces was only queueing one IO per NVMe device. With current patches queuing is fixed and performance is much better. I think this was fixed sometime in 2023. I’m pretty sure both NTFS and ReFS were affected.
      - jiggawatts 3y ago
        Ah, that would explain the absurdly bad I/O performance I was seeing in Azure VMs that had the new NVMe virtual disk controllers! I had spoken with some of the teams involved and they were rather cagey about the root cause, but at least one person mentioned that there were some fixes in the pipeline for Windows Server 2022 NVMe support. I guess this must have been it!
  - hsbauauvhabzb 3y ago
    > [1] “It works for me on my two disk mirror” is an anecdote, not data. While that statement might well be correct that the quote is in fact an anecdote, the following is also an anecdote: ‘From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong.’
    - nisa 3y ago
      Made the mistake using btrfs for a Hadoop cluster at university in kernel 4.x times after reading that SLES uses it and after reading an interview on lwn with someone important, I think the maintainer at that time - that deemed it stable. This must be 10 or 12 years ago or so and it was a wild ride - crashes, manual recovery on 200 machines using clusterssh to get the partitions to mount again. Got out of disk space errors on a 16tb raid 1 (which is not a real raid1) with 5% usage - lot's of sweat I'd rather avoid. Should have just used ext4 in hindsight. For me I decided to not touch it anymore after that experience. I'm sure there is a name for that bias but I don't care. Got burned badly. Lots of people had probably similar experiences and that's were that coming from. Reading the mailing list archives at that time might also be useful to convince yourself that it was more than anecdote.
      - hsbauauvhabzb 3y ago
        I’m not disputing any discourse relating to the factual in/correctness of the anecdote, I’m pointing out that gp is providing an anecdote while disputing anecdotes that they don’t agree with. Provide actual data that’s recent. Linux 4.x was what, 10 years ago? Cars are substantially safer now than they were 10/20/50 years ago, so whose to say your experience with a file system would be different?
        Haemm0r 3y ago
        Same could be said about cars: Why ever buy a [insert brand] again after you've been burned by its reliabity or other issues? You probably just don't, as the alternatives are good and plenty.
  - gerdesj 3y ago
    One place where ReFS is rather decent is "reflinks" - that's where identical blocks are stored once and in the background the rest are simply links to the one block. That is rather useful in backup systems. XFS also supports reflinks amongst other things and is way older than ReFS and hence considered out of beta (which ReFS isn't, by me) I don't trust data to RefS yet - its a fun project that will no doubt prove itself one day. For now, Windows boxes run NTFS and Linux runs ext4 or XFS.
    - Gabrys1 3y ago
      What happens if the very important directory you copied 11 times (just to be sure) ends up producing the same block and doesn't indeed get duplicated as you expected? And now, that block gets corrupted...
      - defrost 3y ago
        Back in the day if I copied (geophysical air) survey data 11 times and put all the copies in the same walk in fire proof safe (in the hanger), that offered no real additional security in the event of a direct hit by an aircraft and explosion while the door was open. If you're going to make 11 copies, they have to go to different physical locations, different devices at least, geographically different places to be sure, or it's pointless. In this instance, block de-duping on a single device makes sense .. expecting mutiple copies on the same device (with or without duplicat block reuse) to offer any additional safety does not.
  - Dylan16807 3y ago
    ReFS only got put back into normal Windows 11 a few months ago. That's a good sign for the future, but it was looking bad for a long time. Also if you turn on data checksums, my understanding is it will delete any file that gets a corrupted sector. And you can only override this behavior on a per-file basis. Unless this changed very recently?
    - MarkSweep 3y ago
      Oh, is it no longer exiled to Windows Pro for Workstations? This feature comparison chart still has this there: https://www.microsoft.com/en-us/windows/business/compare-windows-11 https://www.microsoft.com/en-us/windows/business/compare-win... For what it’s worth, regular Windows 10 & 11 Pro (and other editions maybe?) have supported reading and writing ReFS this whole time. It’s just the option to create a new volume that’s been disabled.
      - marwis 3y ago
        It still sort of is but you can create Dev Drive which is based on ReFS
  - frankjr 3y ago
    > From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong The list cannot be crazy long if Synology uses it for their NASes.
    - yjftsjthsd-h 3y ago
      > The list cannot be crazy long if Synology uses it for their NASes. Synology uses a hybrid BTRFS+mdadm arrangement specifically to deal with reliability problems with BTRFS RAID: https://kb.synology.com/en-us/DSM/tutorial/What_was_the_RAID_implementation_for_Btrfs_File_System_on_SynologyNAS https://kb.synology.com/en-us/DSM/tutorial/What_was_the_RAID...
      - wolletd 3y ago
        Which is kind of the point. BTRFS only has issues with RAID5/6 configurations. Using it as a filesystem for a single disk or partition should be totally fine.
        Dalewyn 3y ago
        Everything I've read about btrfs's RAID5/6 deficiencies is that it can't tolerate sudden losses of power (aka write hole problem), which I think is fine so long as you are aware of it and implement appropriate safety measures such as a UPS or APU. And besides, if you are doing RAID you are probably concerned with the system's uptime which probably means you will have implemented such measures anyway. Note that, yes, I'm aware most home users either aren't aware (nobody RTFM) or are too lazy/cheap to buy a UPS from Office Depot. So perhaps btrfs is warning people to save them from themselves.
        dark-star 3y ago
        A UPS will not much improve the reliability against sudden power loss. At least here in Europe it is much more likely that a PSU or other component fails than that the power line is suddenly interrupted. And lost writes are a problem that all filesystems have. I recommend reading the paper "Parity Lost and Parity Regained" by Krioukov at USENIX 08...
  - yjftsjthsd-h 3y ago
    Weirdly, it's possible that this version could be more stable/reliable/safe than the Linux version, since it's apparently a wholly independent reimplementation. I suppose it depends on whether BTRFS's problems stem from the underlying data format or the actual code as written for the Linux driver.
  - dark-star 3y ago
    ReFS is terrible. We have seen so many customers lose data on ReFS that I started strongly advising everyone against using it. One example: If you (accidentally or on purpose) attach a ReFS disk or LUN to a newer Windows version, it will be silently upgraded to a new ReFS version without any feedback (or chance to prevent it) for the user. No way of attaching the disk on an older Windows version afterwards. But that is not the real problem. The real problem is that the upgrade runs as a separate (user-space) process. If this process crashes or your PC crashes or reboots while it runs, your data is gone. There is no feedback how long it still has to run (we've seen multiple days on large volumes) So yeah, maybe avoid ReFS for a few more years...
  - temac 3y ago
    Advising ReFS is a little bit insane though. I would certainly not entrust it for my data either.
- nyanpasu64 3y ago
  One time I accidentally ran a Visual Studio build in a btrfs git clone rather than my main NTFS drive. By the time I noticed and cancelled the build, there were two folders with an identical name but different contents, which I had to delete the folder name twice. I'd say the driver has issues with concurrency.
  - Kwpolska 3y ago
    I once ran a `git clone` from WSL1 on the C: drive, and tried to build a C++ project in VS. It complained that "EXAMPLE.H" was not found. An "example.h" file did exist in the repo, and my code asked for "example.h". Turns out WSL1 set some obscure bit not known in Win32 land (but enforced by NTFS) that makes the file names case-sensitive, while VS's path normalisation expects a case-insensitive file system. Perhaps this was related to your issue?
    - nyanpasu64 3y ago
      In a separate occasion, I also got that issue (but worse). I once marked a NTFS folder as case-sensitive to help root out all case mismatch bugs (to get a C++ project eventually building on Linux), but then Visual Studio and CMake started spitting out "file not found" errors even for the correct case! I had somehow produced a "cursed" folder that could not be used for building code until I copied (not moved) its contents to a regular case-insensitive NTFS folder.
- summermusic 3y ago
  I have run this casually on my main machine for a few years now. I have a Windows partition, a Linux partition (btrfs on LUKS), and a third btrfs partition where I kept my files. I don’t use it often, but when I do I don’t even notice it. It’s as if Windows could just natively read btrfs all along. This was without any “advanced” usage beyond simply accessing, modifying, or deleting files.
- Ciantic 3y ago
  Heads up, installing both WinBTRFS and OpenZFS on Windows may have problems: "Win OpenZFS driver and WinBtrfs driver dont play well with each other" https://github.com/openzfsonwindows/openzfs/issues/364 https://github.com/openzfsonwindows/openzfs/issues/364
fsiefken 3y ago
Would this make it possible to boot Windows10 and 11 from a btrfs formatted windows usb stick?
- Cu3PO42 3y ago
  Not in its own. You also need a different boot loader. The author has an implementation called Quibble [0] that also supports btrfs. [0] https://github.com/maharmstone/quibble https://github.com/maharmstone/quibble
- Modified3019 3y ago
  You can use Rufus to install 10/11 on a usb SATA/NVMe drive enclosure as “Windows To Go”. In practice it works out pretty decently in my experience using it with windows 10 daily for a while, with a few caveats: 1. You need a stable usb connection 2. You need a usb drive enclosure with a controller chip that is stable/doesn’t overheat 3. Your drive should be powerless resistant. Unfortunately there’s no resource I know of that evaluates power loss handing. Some drives will have a bad time having power suddenly cut. I’ve had good experience with Intel enterprise sata SSD’s and NVMe drives in a Dockcase with capacitor. If your drive stops showing up, a power cycle might help: https://dfarq.homeip.net/fix-dead-ssd/ https://dfarq.homeip.net/fix-dead-ssd/ 4. Have automatic backups setup. Very useful for performance testing and hardware firmware updates that are windows only. When switching between computers, I’ll often have to boot, windows gets confused and then reboot. After that it works. However, I have no experience trying to make use of WinBTRFS or the separate bootloader project, which is apparently currently broken since a few months ago. Ventoy booting a windows VHD file might also be a decent option
  - ajolly 3y ago
    Does windows to go give you the same benefits of doing it on vhdx files (you can do snapshots and roll back state easily) I've got a testing install of Windows on my main hard drive that boots from VHDX for these reasons
    - Modified3019 3y ago
gertop 3y ago
I recommend that everybody reads the README. The author answers all the questions I've had - and much more!
- gerdesj 3y ago
  [flagged]
develatio 3y ago
How come this has "basic and advanced" (whatever that means ¿?) RAID 5/6, while BTRFS itself doesn't? (https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid56-status-and-recommended-practices https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid5...)
- gerdesj 3y ago
  [flagged]
- dark-star 3y ago
  They call RAID0/1/10 "basic" RAID and RAID5/6 "advanced" RAID. I have no idea why. Maybe because the former doesn't require "advanced" parity calculations or something.
- viraptor 3y ago
  That's not quite right. Linux btrfs supports raid5 in general, but has known edge cases which make it not safe to use. Basically it's "available, but experimental, for developers only". Winbtrfs only says the raid5 mode is one of the features, but doesn't really address how well it works. The questions in a related issue (https://github.com/maharmstone/btrfs/issues/293 https://github.com/maharmstone/btrfs/issues/293) have been closed without real answers. I wouldn't risk raid 5/6 on it without getting good answers about the status / testing from the developers first.
  - hsbauauvhabzb 3y ago
    I was under the impression that brtfs didn’t support raid, but could be deployed on top of software raid?
    - _flux 3y ago
      Well, that's the wrong impression. Here's some Arch documentation about it, basically you just create a btrfs on top of multiple devices and it works (to some extent with raid5/raid6 as well): https://wiki.archlinux.org/title/Btrfs#Multi-device_file_system https://wiki.archlinux.org/title/Btrfs#Multi-device_file_sys... . Raid1 apparently works fine. So if you want raid5/6, deploying on top of md is the better option.
dang 3y ago
Related: WinBtrfs – A Windows driver for the next-generation Linux filesystem Btrfs - https://news.ycombinator.com/item?id=15177002 https://news.ycombinator.com/item?id=15177002 - Sept 2017 (100 comments) WinBtrfs v0.7 - https://news.ycombinator.com/item?id=12794214 https://news.ycombinator.com/item?id=12794214 - Oct 2016 (1 comment)
BirAdam 3y ago
It's really awesome that this was a complete reimplementation with no Linux code, and it's additionally awesome that this is available for both XP/2k3 and ReactOS. I will have to try it out on one of my older machines :-)
- userbinator 3y ago
  and it's additionally awesome that this is available for both XP/2k3 and ReactOS ReactOS is supposed to be API-compatible with Windows, so that's not too surprising.
  - pxc 3y ago
    It's not surprising, but it is really nice that this gives ReactOS a nice, modern CoW filesystem.
- jauntywundrkind 3y ago
  One of the interesting patterns happening in Rust is io-less libraries. I'm not sure where best to link this phenomenon. It here s a open issue for an io-less quic library, from 2019, https://github.com/aiortc/aioquic/issues/4 https://github.com/aiortc/aioquic/issues/4 It'd be so fracking sweet to see filesystems follow this pattern. If we could re-use the file system logic, but apply it to windows or fuse or Linux or wasm linearly-addressed-storage, that would allow such intensely cool forms of portability/reuse & bending/hacking.
  - unshavedyak 3y ago
    How is this implemented in practice? Special care to keep io on the outermost layers? Never thought about software in this way. Seems really tough, but interesting Wonder how well it scales to larger applications. Ie is there a codesize where io-less becomes too difficult? Perhaps performance concerns? Hmm
    - Arnavion 3y ago
      It's not really an "application" thing. It's meant to be a design for libraries that implement protocols of some sort. All the library API acts on byte buffers and leaves the network socket etc stuff to the library user. So when the library needs to write data to a socket, the API instead returns a byte buffer to the caller, and the caller writes it to the network socket. When the library needs to read data from a socket, it instead expects the caller to do that and then give the populated byte buffer to a library function to ingest it. Also, quite the opposite, it's *easier* to design a library this way because it's strictly less code the library needs to contain. Specifically in Rust it also has the advantage that the library becomes agnostic to sync vs async I/O since that's handled by the library user. Correspondingly, it is slightly harder for the library user to use such a library, but it's usually just a matter of writing a tiny generic wrapper around the network socket type to connect it to the library functions.
westurner 3y ago
> See also Quibble, an experimental bootloader allowing Windows to boot from Btrfs, and Ntfs2btrfs, a tool which allows in-place conversion of NTFS filesystems. The chocolatey package for WinBtrfs: https://community.chocolatey.org/packages/winbtrfs https://community.chocolatey.org/packages/winbtrfs
rustcleaner 3y ago
Should have been ZFS. :*^(
- Fnoord 3y ago
  Exists! [1] ZFS seems to be the most cross-platform of the modern filesystems. Although there's a Paragon driver for APFS for Windows and a FOSS driver for native Linux APFS as well as one for FUSE. Personally, I keep track of bcachefs which got merged in Linux 6.7. But it won't be cross-platform. [1] https://github.com/openzfsonwindows/openzfs https://github.com/openzfsonwindows/openzfs [2] https://bcachefs.org https://bcachefs.org
  - different_base 3y ago
    What is the fuss about bcachefs? What does it do better compared to Btrfs or Zfs?
    - lproven 3y ago
      Its tagline, from the website: https://bcachefs.org/ https://bcachefs.org/ ... is: "The COW filesystem for Linux that won't eat your data". This is, IMHO, a dig at Btrfs, which is famous for recovery problems. Btrfs doens't have a working reliable `fsck` replacement or a working `df` command which gives reliable results. So, bcachefs aims to be better than Btrfs and that's a low bar to clear. It's "better" than ZFS because it's GPL and designed to do the same as ZFS but be integrated into the Linux kernel, which ZFS can't be because its license doesn't allow it. For this it merely needs to equal ZFS' features and functions, not exceed them.
graphe 3y ago
What is the purpose of using this in production? I thought people just ssh into Linux if you need it to just work. For my own purposes I used to use an ext3 driver on Win7, never failed on me, just switched to Linux.
- yjftsjthsd-h 3y ago
  Some people want to access the same data volume from Linux and Windows (see the person dual-booting upthread).
  - Zambyte 3y ago
    You can also just pass the partition to a VM and access the VM storage however you want. I would trust that a lot more than this to be honest. Nothing against this project in particular, I just don't find the idea of using a filesystem driver on Windows to access a filesystem that Windows doesn't normally support. I don't really trust Windows to handle that well :P
nukeyoular 3y ago
[dead]
minroot 3y ago
I tried to use it it few weeks ago on a btrfs hard drive. But i couldn't make it work. Then i used wsl to access it. Worked for a few run but then things just started to fail. It wouldn't even get mounted. Then I realized i can just boot a live iso of linux and copy/move files to windows drive and to the btrfs drive. That what i am doing now, using Fedora Workstation live iso on USB drive with ventoy.
- josteink 3y ago
  Sounds like an authentic experience. Now you can lose data to btrfs on Windows too :-D
qwerty456127 3y ago
Why still use hardware RAID nowadays when we have BTRFS and ZFS?
- lm411 3y ago
  Performance, reliability, and BBU or CacheVault. Hardware RAID is worth the money when uptime and performance are important. I've seen good RAID cards keep a server running where native direct SATA would have brought the server down.
  - Brian_K_White 3y ago
    As someone who used to use both in production for many years, I do not miss the days of hardware raid. No contest. Would never go back. The fancier and more expensive the worse. The advantages were always theoretical and the disadvantages were always real. It caused way more problems than it solved or prevented, and the problems are worse because you are essentially powerless to address them. With software raid you have the control to address problems even when you fall off the happy path, and infinite flexibility wrt hardware and emergency recovery. In macro, hardware raid is less reliable, not more. Performance is no better than a draw, not better.
  - ndsipa_pomu 3y ago
    The problems with "hardware" RAID are the proprietary disk formats and the need to buy multiple RAID cards in case one fails, as you may need to have the same vendor/version for it to be a straight swap and still keep all your data. There can also be issues with drivers, especially if the "hardware" RAID is partly implemented by the driver. I've had issues in the past with needing to put hardward RAID drivers into the initramfs of Linux boxes just so that they can boot. With software RAID, you can just plug the disks into other servers without those kinds of problems.
  - cheema33 3y ago
    > Hardware RAID is worth the money when uptime and performance are important. I think this is no longer true. Software RAID is now better in almost every way, including performance.
mgaunard 3y ago
I feel like btrfs has been in development since forever and getting no adoption at all. When is the year of the btrfs file system coming?
- 0dayz 3y ago
  It's already the default on opensuse and fedora while Ubuntu, Debian arch and Gentoo all support it so it's not that hard to adopt. The issue is that btrfs still does not run perfectly rock solid on raid other than 0 and 1. Plus it's compression and performance is still far off what alternatives can provide (but this is also getting better).
- int_19h 3y ago
  It's the default on all current Synology NAS boxes, and has been for quite a while.
poisonborz 3y ago
Wanted to use it for a while but a glance at the github issues was enough to nope out. BSODs, lockups, usage spikes, corruption. I so much wish for a stable btrfs/zfs driver, I'd gladly throw my credit card at it. I don't get why these things don't get more traction.
- cies 3y ago
  Because it's not supported by MS, and the driver makers cannot read the code if window's kernel. So even when the problems you mentioned are fixed, you will probably still not be able to boot windows from btrfs. When in Rome do as the Romans do. Maybe we should only run windows virtualized :)
  - mlfreeman 3y ago
    > you will probably still not be able to boot windows from btrfs. See his other repo, he appears to be working on that. https://github.com/maharmstone/quibble https://github.com/maharmstone/quibble
  - VHRanger 3y ago
    > Because it's not supported by MS, and the driver makers cannot read the code if window's kernel I mean, they can't admit to reading it. But that source code is out there since the leak isn't it?
- beeboobaa3 3y ago
  > I'd gladly throw my credit card at it https://github.com/maharmstone/btrfs?tab=readme-ov-file#donations https://github.com/maharmstone/btrfs?tab=readme-ov-file#dona...
jcd000 3y ago
I dual boot and have been using this for a while now (the older version). It does work, but some problems are to be expected. While impressive, it is not production-level. For me that's fine since I boot windows pretty rarely, but probably not for everyone. I would love to see that the new version works with fewer problems.
indiebat 3y ago
Using this driver for a while now on laptop, I use it to both code as part of work (on linux) and play (games, movies and other media on windows) Anyone contemplating use and rattled about data corruption etc on btrfs partitions and drives, focus on mount options in README 1. use `Ignore` for (arch)linux system partition 2. use `Readonly` for everything else That being said, I never faced cpu spikes/corruption using the driver on 20TB external HDD with complete btrfs zstd:2 compression mount
Already__Taken 3y ago
I used this and btrfs on the Steam deck to preload my library because the network would have taken forever and the internet I had at the time would have taken that x100. This could be the first FS that just-works across nix, Mac and windows since fat.
- oynqr 3y ago
  Everybody always forgets about UDF...
  - globular-toast 3y ago
    Hmm... Is there any reason why this isn't more widely used for usb sticks? I must admit I never considered it as I always thought of it as the read-only DVD filesystem, but turns out it's a full read/write fs.

311 points by frankjr 3y ago

[dead]

Awesome! Anyone using this long term or in production who can comment on how it's been working? I see TRIM is supported. Is RETRIM also (or whatever is needed during drive optimization to release areas that didn't get TRIMmed the first time due to a full command queue). Could this serve as an effective NTFS replacement with data parity for those who don't like ReFS? How mature is it compared to ZFS on Windows?

deleted 3y ago
[deleted]
jiggawatts 3y ago
ReFS with Storage Spaces already serves this purpose and is integrated and fully supported. From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong.[1] Using something that is unreliable at best on its native OS shoehorned into Windows is madness. Fun for a lark, sure, but I would never ever entrust this combination with any actual data. [1] “It works for me on my two disk mirror” is an anecdote, not data.
- rkagerer 3y ago
  Thanks. I tried ReFS when it first came out and it was terribly slow (with data parity on), and Storage Spaces was obscure to set up and manage. Has the landscape improved?
  - mgerdts 3y ago
    On WS2022 without patches I noticed that Storage Spaces was only queueing one IO per NVMe device. With current patches queuing is fixed and performance is much better. I think this was fixed sometime in 2023. I’m pretty sure both NTFS and ReFS were affected.
    - jiggawatts 3y ago
      Ah, that would explain the absurdly bad I/O performance I was seeing in Azure VMs that had the new NVMe virtual disk controllers! I had spoken with some of the teams involved and they were rather cagey about the root cause, but at least one person mentioned that there were some fixes in the pipeline for Windows Server 2022 NVMe support. I guess this must have been it!
- hsbauauvhabzb 3y ago
  > [1] “It works for me on my two disk mirror” is an anecdote, not data. While that statement might well be correct that the quote is in fact an anecdote, the following is also an anecdote: ‘From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong.’
  - nisa 3y ago
    Made the mistake using btrfs for a Hadoop cluster at university in kernel 4.x times after reading that SLES uses it and after reading an interview on lwn with someone important, I think the maintainer at that time - that deemed it stable. This must be 10 or 12 years ago or so and it was a wild ride - crashes, manual recovery on 200 machines using clusterssh to get the partitions to mount again. Got out of disk space errors on a 16tb raid 1 (which is not a real raid1) with 5% usage - lot's of sweat I'd rather avoid. Should have just used ext4 in hindsight. For me I decided to not touch it anymore after that experience. I'm sure there is a name for that bias but I don't care. Got burned badly. Lots of people had probably similar experiences and that's were that coming from. Reading the mailing list archives at that time might also be useful to convince yourself that it was more than anecdote.
    - hsbauauvhabzb 3y ago
      I’m not disputing any discourse relating to the factual in/correctness of the anecdote, I’m pointing out that gp is providing an anecdote while disputing anecdotes that they don’t agree with. Provide actual data that’s recent. Linux 4.x was what, 10 years ago? Cars are substantially safer now than they were 10/20/50 years ago, so whose to say your experience with a file system would be different?
      - Haemm0r 3y ago
        Same could be said about cars: Why ever buy a [insert brand] again after you've been burned by its reliabity or other issues? You probably just don't, as the alternatives are good and plenty.
- gerdesj 3y ago
  One place where ReFS is rather decent is "reflinks" - that's where identical blocks are stored once and in the background the rest are simply links to the one block. That is rather useful in backup systems. XFS also supports reflinks amongst other things and is way older than ReFS and hence considered out of beta (which ReFS isn't, by me) I don't trust data to RefS yet - its a fun project that will no doubt prove itself one day. For now, Windows boxes run NTFS and Linux runs ext4 or XFS.
  - Gabrys1 3y ago
    What happens if the very important directory you copied 11 times (just to be sure) ends up producing the same block and doesn't indeed get duplicated as you expected? And now, that block gets corrupted...
    - defrost 3y ago
      Back in the day if I copied (geophysical air) survey data 11 times and put all the copies in the same walk in fire proof safe (in the hanger), that offered no real additional security in the event of a direct hit by an aircraft and explosion while the door was open. If you're going to make 11 copies, they have to go to different physical locations, different devices at least, geographically different places to be sure, or it's pointless. In this instance, block de-duping on a single device makes sense .. expecting mutiple copies on the same device (with or without duplicat block reuse) to offer any additional safety does not.
- Dylan16807 3y ago
  ReFS only got put back into normal Windows 11 a few months ago. That's a good sign for the future, but it was looking bad for a long time. Also if you turn on data checksums, my understanding is it will delete any file that gets a corrupted sector. And you can only override this behavior on a per-file basis. Unless this changed very recently?
  - MarkSweep 3y ago
    Oh, is it no longer exiled to Windows Pro for Workstations? This feature comparison chart still has this there: https://www.microsoft.com/en-us/windows/business/compare-windows-11 https://www.microsoft.com/en-us/windows/business/compare-win... For what it’s worth, regular Windows 10 & 11 Pro (and other editions maybe?) have supported reading and writing ReFS this whole time. It’s just the option to create a new volume that’s been disabled.
    - marwis 3y ago
      It still sort of is but you can create Dev Drive which is based on ReFS
- frankjr 3y ago
  > From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong The list cannot be crazy long if Synology uses it for their NASes.
  - yjftsjthsd-h 3y ago
    > The list cannot be crazy long if Synology uses it for their NASes. Synology uses a hybrid BTRFS+mdadm arrangement specifically to deal with reliability problems with BTRFS RAID: https://kb.synology.com/en-us/DSM/tutorial/What_was_the_RAID_implementation_for_Btrfs_File_System_on_SynologyNAS https://kb.synology.com/en-us/DSM/tutorial/What_was_the_RAID...
    - wolletd 3y ago
      Which is kind of the point. BTRFS only has issues with RAID5/6 configurations. Using it as a filesystem for a single disk or partition should be totally fine.
      - Dalewyn 3y ago
        Everything I've read about btrfs's RAID5/6 deficiencies is that it can't tolerate sudden losses of power (aka write hole problem), which I think is fine so long as you are aware of it and implement appropriate safety measures such as a UPS or APU. And besides, if you are doing RAID you are probably concerned with the system's uptime which probably means you will have implemented such measures anyway. Note that, yes, I'm aware most home users either aren't aware (nobody RTFM) or are too lazy/cheap to buy a UPS from Office Depot. So perhaps btrfs is warning people to save them from themselves.
        dark-star 3y ago
        A UPS will not much improve the reliability against sudden power loss. At least here in Europe it is much more likely that a PSU or other component fails than that the power line is suddenly interrupted. And lost writes are a problem that all filesystems have. I recommend reading the paper "Parity Lost and Parity Regained" by Krioukov at USENIX 08...
- yjftsjthsd-h 3y ago
  Weirdly, it's possible that this version could be more stable/reliable/safe than the Linux version, since it's apparently a wholly independent reimplementation. I suppose it depends on whether BTRFS's problems stem from the underlying data format or the actual code as written for the Linux driver.
- dark-star 3y ago
  ReFS is terrible. We have seen so many customers lose data on ReFS that I started strongly advising everyone against using it. One example: If you (accidentally or on purpose) attach a ReFS disk or LUN to a newer Windows version, it will be silently upgraded to a new ReFS version without any feedback (or chance to prevent it) for the user. No way of attaching the disk on an older Windows version afterwards. But that is not the real problem. The real problem is that the upgrade runs as a separate (user-space) process. If this process crashes or your PC crashes or reboots while it runs, your data is gone. There is no feedback how long it still has to run (we've seen multiple days on large volumes) So yeah, maybe avoid ReFS for a few more years...
- temac 3y ago
  Advising ReFS is a little bit insane though. I would certainly not entrust it for my data either.
nyanpasu64 3y ago
One time I accidentally ran a Visual Studio build in a btrfs git clone rather than my main NTFS drive. By the time I noticed and cancelled the build, there were two folders with an identical name but different contents, which I had to delete the folder name twice. I'd say the driver has issues with concurrency.
- Kwpolska 3y ago
  I once ran a `git clone` from WSL1 on the C: drive, and tried to build a C++ project in VS. It complained that "EXAMPLE.H" was not found. An "example.h" file did exist in the repo, and my code asked for "example.h". Turns out WSL1 set some obscure bit not known in Win32 land (but enforced by NTFS) that makes the file names case-sensitive, while VS's path normalisation expects a case-insensitive file system. Perhaps this was related to your issue?
  - nyanpasu64 3y ago
    In a separate occasion, I also got that issue (but worse). I once marked a NTFS folder as case-sensitive to help root out all case mismatch bugs (to get a C++ project eventually building on Linux), but then Visual Studio and CMake started spitting out "file not found" errors even for the correct case! I had somehow produced a "cursed" folder that could not be used for building code until I copied (not moved) its contents to a regular case-insensitive NTFS folder.
summermusic 3y ago
I have run this casually on my main machine for a few years now. I have a Windows partition, a Linux partition (btrfs on LUKS), and a third btrfs partition where I kept my files. I don’t use it often, but when I do I don’t even notice it. It’s as if Windows could just natively read btrfs all along. This was without any “advanced” usage beyond simply accessing, modifying, or deleting files.
Ciantic 3y ago
Heads up, installing both WinBTRFS and OpenZFS on Windows may have problems: "Win OpenZFS driver and WinBtrfs driver dont play well with each other" https://github.com/openzfsonwindows/openzfs/issues/364 https://github.com/openzfsonwindows/openzfs/issues/364

fsiefken 3y ago

Would this make it possible to boot Windows10 and 11 from a btrfs formatted windows usb stick?

Cu3PO42 3y ago
Not in its own. You also need a different boot loader. The author has an implementation called Quibble [0] that also supports btrfs. [0] https://github.com/maharmstone/quibble https://github.com/maharmstone/quibble
Modified3019 3y ago
You can use Rufus to install 10/11 on a usb SATA/NVMe drive enclosure as “Windows To Go”. In practice it works out pretty decently in my experience using it with windows 10 daily for a while, with a few caveats: 1. You need a stable usb connection 2. You need a usb drive enclosure with a controller chip that is stable/doesn’t overheat 3. Your drive should be powerless resistant. Unfortunately there’s no resource I know of that evaluates power loss handing. Some drives will have a bad time having power suddenly cut. I’ve had good experience with Intel enterprise sata SSD’s and NVMe drives in a Dockcase with capacitor. If your drive stops showing up, a power cycle might help: https://dfarq.homeip.net/fix-dead-ssd/ https://dfarq.homeip.net/fix-dead-ssd/ 4. Have automatic backups setup. Very useful for performance testing and hardware firmware updates that are windows only. When switching between computers, I’ll often have to boot, windows gets confused and then reboot. After that it works. However, I have no experience trying to make use of WinBTRFS or the separate bootloader project, which is apparently currently broken since a few months ago. Ventoy booting a windows VHD file might also be a decent option
- ajolly 3y ago
  Does windows to go give you the same benefits of doing it on vhdx files (you can do snapshots and roll back state easily) I've got a testing install of Windows on my main hard drive that boots from VHDX for these reasons
  - Modified3019 3y ago

gertop 3y ago

I recommend that everybody reads the README. The author answers all the questions I've had - and much more!

gerdesj 3y ago
[flagged]

develatio 3y ago

How come this has "basic and advanced" (whatever that means ¿?) RAID 5/6, while BTRFS itself doesn't? (https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid56-status-and-recommended-practices https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid5...)

gerdesj 3y ago
[flagged]
dark-star 3y ago
They call RAID0/1/10 "basic" RAID and RAID5/6 "advanced" RAID. I have no idea why. Maybe because the former doesn't require "advanced" parity calculations or something.
viraptor 3y ago
That's not quite right. Linux btrfs supports raid5 in general, but has known edge cases which make it not safe to use. Basically it's "available, but experimental, for developers only". Winbtrfs only says the raid5 mode is one of the features, but doesn't really address how well it works. The questions in a related issue (https://github.com/maharmstone/btrfs/issues/293 https://github.com/maharmstone/btrfs/issues/293) have been closed without real answers. I wouldn't risk raid 5/6 on it without getting good answers about the status / testing from the developers first.
- hsbauauvhabzb 3y ago
  I was under the impression that brtfs didn’t support raid, but could be deployed on top of software raid?
  - _flux 3y ago
    Well, that's the wrong impression. Here's some Arch documentation about it, basically you just create a btrfs on top of multiple devices and it works (to some extent with raid5/raid6 as well): https://wiki.archlinux.org/title/Btrfs#Multi-device_file_system https://wiki.archlinux.org/title/Btrfs#Multi-device_file_sys... . Raid1 apparently works fine. So if you want raid5/6, deploying on top of md is the better option.

dang 3y ago

Related: WinBtrfs – A Windows driver for the next-generation Linux filesystem Btrfs - https://news.ycombinator.com/item?id=15177002 https://news.ycombinator.com/item?id=15177002 - Sept 2017 (100 comments) WinBtrfs v0.7 - https://news.ycombinator.com/item?id=12794214 https://news.ycombinator.com/item?id=12794214 - Oct 2016 (1 comment)

BirAdam 3y ago

It's really awesome that this was a complete reimplementation with no Linux code, and it's additionally awesome that this is available for both XP/2k3 and ReactOS. I will have to try it out on one of my older machines :-)

userbinator 3y ago
and it's additionally awesome that this is available for both XP/2k3 and ReactOS ReactOS is supposed to be API-compatible with Windows, so that's not too surprising.
- pxc 3y ago
  It's not surprising, but it is really nice that this gives ReactOS a nice, modern CoW filesystem.
jauntywundrkind 3y ago
One of the interesting patterns happening in Rust is io-less libraries. I'm not sure where best to link this phenomenon. It here s a open issue for an io-less quic library, from 2019, https://github.com/aiortc/aioquic/issues/4 https://github.com/aiortc/aioquic/issues/4 It'd be so fracking sweet to see filesystems follow this pattern. If we could re-use the file system logic, but apply it to windows or fuse or Linux or wasm linearly-addressed-storage, that would allow such intensely cool forms of portability/reuse & bending/hacking.
- unshavedyak 3y ago
  How is this implemented in practice? Special care to keep io on the outermost layers? Never thought about software in this way. Seems really tough, but interesting Wonder how well it scales to larger applications. Ie is there a codesize where io-less becomes too difficult? Perhaps performance concerns? Hmm
  - Arnavion 3y ago
    It's not really an "application" thing. It's meant to be a design for libraries that implement protocols of some sort. All the library API acts on byte buffers and leaves the network socket etc stuff to the library user. So when the library needs to write data to a socket, the API instead returns a byte buffer to the caller, and the caller writes it to the network socket. When the library needs to read data from a socket, it instead expects the caller to do that and then give the populated byte buffer to a library function to ingest it. Also, quite the opposite, it's *easier* to design a library this way because it's strictly less code the library needs to contain. Specifically in Rust it also has the advantage that the library becomes agnostic to sync vs async I/O since that's handled by the library user. Correspondingly, it is slightly harder for the library user to use such a library, but it's usually just a matter of writing a tiny generic wrapper around the network socket type to connect it to the library functions.

westurner 3y ago

> See also Quibble, an experimental bootloader allowing Windows to boot from Btrfs, and Ntfs2btrfs, a tool which allows in-place conversion of NTFS filesystems. The chocolatey package for WinBtrfs: https://community.chocolatey.org/packages/winbtrfs https://community.chocolatey.org/packages/winbtrfs

rustcleaner 3y ago

Should have been ZFS. :*^(

Fnoord 3y ago
Exists! [1] ZFS seems to be the most cross-platform of the modern filesystems. Although there's a Paragon driver for APFS for Windows and a FOSS driver for native Linux APFS as well as one for FUSE. Personally, I keep track of bcachefs which got merged in Linux 6.7. But it won't be cross-platform. [1] https://github.com/openzfsonwindows/openzfs https://github.com/openzfsonwindows/openzfs [2] https://bcachefs.org https://bcachefs.org
- different_base 3y ago
  What is the fuss about bcachefs? What does it do better compared to Btrfs or Zfs?
  - lproven 3y ago
    Its tagline, from the website: https://bcachefs.org/ https://bcachefs.org/ ... is: "The COW filesystem for Linux that won't eat your data". This is, IMHO, a dig at Btrfs, which is famous for recovery problems. Btrfs doens't have a working reliable `fsck` replacement or a working `df` command which gives reliable results. So, bcachefs aims to be better than Btrfs and that's a low bar to clear. It's "better" than ZFS because it's GPL and designed to do the same as ZFS but be integrated into the Linux kernel, which ZFS can't be because its license doesn't allow it. For this it merely needs to equal ZFS' features and functions, not exceed them.

graphe 3y ago

What is the purpose of using this in production? I thought people just ssh into Linux if you need it to just work. For my own purposes I used to use an ext3 driver on Win7, never failed on me, just switched to Linux.

yjftsjthsd-h 3y ago
Some people want to access the same data volume from Linux and Windows (see the person dual-booting upthread).
- Zambyte 3y ago
  You can also just pass the partition to a VM and access the VM storage however you want. I would trust that a lot more than this to be honest. Nothing against this project in particular, I just don't find the idea of using a filesystem driver on Windows to access a filesystem that Windows doesn't normally support. I don't really trust Windows to handle that well :P

nukeyoular 3y ago

[dead]

minroot 3y ago

I tried to use it it few weeks ago on a btrfs hard drive. But i couldn't make it work. Then i used wsl to access it. Worked for a few run but then things just started to fail. It wouldn't even get mounted. Then I realized i can just boot a live iso of linux and copy/move files to windows drive and to the btrfs drive. That what i am doing now, using Fedora Workstation live iso on USB drive with ventoy.

josteink 3y ago
Sounds like an authentic experience. Now you can lose data to btrfs on Windows too :-D

qwerty456127 3y ago

Why still use hardware RAID nowadays when we have BTRFS and ZFS?

lm411 3y ago
Performance, reliability, and BBU or CacheVault. Hardware RAID is worth the money when uptime and performance are important. I've seen good RAID cards keep a server running where native direct SATA would have brought the server down.
- Brian_K_White 3y ago
  As someone who used to use both in production for many years, I do not miss the days of hardware raid. No contest. Would never go back. The fancier and more expensive the worse. The advantages were always theoretical and the disadvantages were always real. It caused way more problems than it solved or prevented, and the problems are worse because you are essentially powerless to address them. With software raid you have the control to address problems even when you fall off the happy path, and infinite flexibility wrt hardware and emergency recovery. In macro, hardware raid is less reliable, not more. Performance is no better than a draw, not better.
- ndsipa_pomu 3y ago
  The problems with "hardware" RAID are the proprietary disk formats and the need to buy multiple RAID cards in case one fails, as you may need to have the same vendor/version for it to be a straight swap and still keep all your data. There can also be issues with drivers, especially if the "hardware" RAID is partly implemented by the driver. I've had issues in the past with needing to put hardward RAID drivers into the initramfs of Linux boxes just so that they can boot. With software RAID, you can just plug the disks into other servers without those kinds of problems.
- cheema33 3y ago
  > Hardware RAID is worth the money when uptime and performance are important. I think this is no longer true. Software RAID is now better in almost every way, including performance.

mgaunard 3y ago

I feel like btrfs has been in development since forever and getting no adoption at all. When is the year of the btrfs file system coming?

0dayz 3y ago
It's already the default on opensuse and fedora while Ubuntu, Debian arch and Gentoo all support it so it's not that hard to adopt. The issue is that btrfs still does not run perfectly rock solid on raid other than 0 and 1. Plus it's compression and performance is still far off what alternatives can provide (but this is also getting better).
int_19h 3y ago
It's the default on all current Synology NAS boxes, and has been for quite a while.

poisonborz 3y ago

Wanted to use it for a while but a glance at the github issues was enough to nope out. BSODs, lockups, usage spikes, corruption. I so much wish for a stable btrfs/zfs driver, I'd gladly throw my credit card at it. I don't get why these things don't get more traction.

cies 3y ago
Because it's not supported by MS, and the driver makers cannot read the code if window's kernel. So even when the problems you mentioned are fixed, you will probably still not be able to boot windows from btrfs. When in Rome do as the Romans do. Maybe we should only run windows virtualized :)
- mlfreeman 3y ago
  > you will probably still not be able to boot windows from btrfs. See his other repo, he appears to be working on that. https://github.com/maharmstone/quibble https://github.com/maharmstone/quibble
- VHRanger 3y ago
  > Because it's not supported by MS, and the driver makers cannot read the code if window's kernel I mean, they can't admit to reading it. But that source code is out there since the leak isn't it?
beeboobaa3 3y ago
> I'd gladly throw my credit card at it https://github.com/maharmstone/btrfs?tab=readme-ov-file#donations https://github.com/maharmstone/btrfs?tab=readme-ov-file#dona...

jcd000 3y ago

I dual boot and have been using this for a while now (the older version). It does work, but some problems are to be expected. While impressive, it is not production-level. For me that's fine since I boot windows pretty rarely, but probably not for everyone. I would love to see that the new version works with fewer problems.

indiebat 3y ago

Using this driver for a while now on laptop, I use it to both code as part of work (on linux) and play (games, movies and other media on windows) Anyone contemplating use and rattled about data corruption etc on btrfs partitions and drives, focus on mount options in README 1. use `Ignore` for (arch)linux system partition 2. use `Readonly` for everything else That being said, I never faced cpu spikes/corruption using the driver on 20TB external HDD with complete btrfs zstd:2 compression mount

Already__Taken 3y ago

I used this and btrfs on the Steam deck to preload my library because the network would have taken forever and the internet I had at the time would have taken that x100. This could be the first FS that just-works across nix, Mac and windows since fat.

oynqr 3y ago
Everybody always forgets about UDF...
- globular-toast 3y ago
  Hmm... Is there any reason why this isn't more widely used for usb sticks? I must admit I never considered it as I always thought of it as the read-only DVD filesystem, but turns out it's a full read/write fs.