19 ms·

Awesome! Anyone using this long term or in production who can comment on how it's been working? I see TRIM is supported. Is RETRIM also (or whatever is neede

Awesome! Anyone using this long term or in production who can comment on how it's been working? I see TRIM is supported. Is RETRIM also (or whatever is needed during drive optimization to release areas that didn't get TRIMmed the first time due to a full command queue). Could this serve as an effective NTFS replacement with data parity for those who don't like ReFS? How mature is it compared to ZFS on Windows?

deleted 3y ago
[deleted]
jiggawatts 3y ago
ReFS with Storage Spaces already serves this purpose and is integrated and fully supported. From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong.[1] Using something that is unreliable at best on its native OS shoehorned into Windows is madness. Fun for a lark, sure, but I would never ever entrust this combination with any actual data. [1] “It works for me on my two disk mirror” is an anecdote, not data.
- rkagerer 3y ago
  Thanks. I tried ReFS when it first came out and it was terribly slow (with data parity on), and Storage Spaces was obscure to set up and manage. Has the landscape improved?
  - mgerdts 3y ago
    On WS2022 without patches I noticed that Storage Spaces was only queueing one IO per NVMe device. With current patches queuing is fixed and performance is much better. I think this was fixed sometime in 2023. I’m pretty sure both NTFS and ReFS were affected.
    - jiggawatts 3y ago
      Ah, that would explain the absurdly bad I/O performance I was seeing in Azure VMs that had the new NVMe virtual disk controllers! I had spoken with some of the teams involved and they were rather cagey about the root cause, but at least one person mentioned that there were some fixes in the pipeline for Windows Server 2022 NVMe support. I guess this must have been it!
- hsbauauvhabzb 3y ago
  > [1] “It works for me on my two disk mirror” is an anecdote, not data. While that statement might well be correct that the quote is in fact an anecdote, the following is also an anecdote: ‘From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong.’
  - nisa 3y ago
    Made the mistake using btrfs for a Hadoop cluster at university in kernel 4.x times after reading that SLES uses it and after reading an interview on lwn with someone important, I think the maintainer at that time - that deemed it stable. This must be 10 or 12 years ago or so and it was a wild ride - crashes, manual recovery on 200 machines using clusterssh to get the partitions to mount again. Got out of disk space errors on a 16tb raid 1 (which is not a real raid1) with 5% usage - lot's of sweat I'd rather avoid. Should have just used ext4 in hindsight. For me I decided to not touch it anymore after that experience. I'm sure there is a name for that bias but I don't care. Got burned badly. Lots of people had probably similar experiences and that's were that coming from. Reading the mailing list archives at that time might also be useful to convince yourself that it was more than anecdote.
    - hsbauauvhabzb 3y ago
      I’m not disputing any discourse relating to the factual in/correctness of the anecdote, I’m pointing out that gp is providing an anecdote while disputing anecdotes that they don’t agree with. Provide actual data that’s recent. Linux 4.x was what, 10 years ago? Cars are substantially safer now than they were 10/20/50 years ago, so whose to say your experience with a file system would be different?
      - Haemm0r 3y ago
        Same could be said about cars: Why ever buy a [insert brand] again after you've been burned by its reliabity or other issues? You probably just don't, as the alternatives are good and plenty.
- gerdesj 3y ago
  One place where ReFS is rather decent is "reflinks" - that's where identical blocks are stored once and in the background the rest are simply links to the one block. That is rather useful in backup systems. XFS also supports reflinks amongst other things and is way older than ReFS and hence considered out of beta (which ReFS isn't, by me) I don't trust data to RefS yet - its a fun project that will no doubt prove itself one day. For now, Windows boxes run NTFS and Linux runs ext4 or XFS.
  - Gabrys1 3y ago
    What happens if the very important directory you copied 11 times (just to be sure) ends up producing the same block and doesn't indeed get duplicated as you expected? And now, that block gets corrupted...
    - defrost 3y ago
      Back in the day if I copied (geophysical air) survey data 11 times and put all the copies in the same walk in fire proof safe (in the hanger), that offered no real additional security in the event of a direct hit by an aircraft and explosion while the door was open. If you're going to make 11 copies, they have to go to different physical locations, different devices at least, geographically different places to be sure, or it's pointless. In this instance, block de-duping on a single device makes sense .. expecting mutiple copies on the same device (with or without duplicat block reuse) to offer any additional safety does not.
- Dylan16807 3y ago
  ReFS only got put back into normal Windows 11 a few months ago. That's a good sign for the future, but it was looking bad for a long time. Also if you turn on data checksums, my understanding is it will delete any file that gets a corrupted sector. And you can only override this behavior on a per-file basis. Unless this changed very recently?
  - MarkSweep 3y ago
    Oh, is it no longer exiled to Windows Pro for Workstations? This feature comparison chart still has this there: https://www.microsoft.com/en-us/windows/business/compare-windows-11 https://www.microsoft.com/en-us/windows/business/compare-win... For what it’s worth, regular Windows 10 & 11 Pro (and other editions maybe?) have supported reading and writing ReFS this whole time. It’s just the option to create a new volume that’s been disabled.
    - marwis 3y ago
      It still sort of is but you can create Dev Drive which is based on ReFS
- frankjr 3y ago
  > From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong The list cannot be crazy long if Synology uses it for their NASes.
  - yjftsjthsd-h 3y ago
    > The list cannot be crazy long if Synology uses it for their NASes. Synology uses a hybrid BTRFS+mdadm arrangement specifically to deal with reliability problems with BTRFS RAID: https://kb.synology.com/en-us/DSM/tutorial/What_was_the_RAID_implementation_for_Btrfs_File_System_on_SynologyNAS https://kb.synology.com/en-us/DSM/tutorial/What_was_the_RAID...
    - wolletd 3y ago
      Which is kind of the point. BTRFS only has issues with RAID5/6 configurations. Using it as a filesystem for a single disk or partition should be totally fine.
      - Dalewyn 3y ago
        Everything I've read about btrfs's RAID5/6 deficiencies is that it can't tolerate sudden losses of power (aka write hole problem), which I think is fine so long as you are aware of it and implement appropriate safety measures such as a UPS or APU. And besides, if you are doing RAID you are probably concerned with the system's uptime which probably means you will have implemented such measures anyway. Note that, yes, I'm aware most home users either aren't aware (nobody RTFM) or are too lazy/cheap to buy a UPS from Office Depot. So perhaps btrfs is warning people to save them from themselves.
        dark-star 3y ago
        A UPS will not much improve the reliability against sudden power loss. At least here in Europe it is much more likely that a PSU or other component fails than that the power line is suddenly interrupted. And lost writes are a problem that all filesystems have. I recommend reading the paper "Parity Lost and Parity Regained" by Krioukov at USENIX 08...
- yjftsjthsd-h 3y ago
  Weirdly, it's possible that this version could be more stable/reliable/safe than the Linux version, since it's apparently a wholly independent reimplementation. I suppose it depends on whether BTRFS's problems stem from the underlying data format or the actual code as written for the Linux driver.
- dark-star 3y ago
  ReFS is terrible. We have seen so many customers lose data on ReFS that I started strongly advising everyone against using it. One example: If you (accidentally or on purpose) attach a ReFS disk or LUN to a newer Windows version, it will be silently upgraded to a new ReFS version without any feedback (or chance to prevent it) for the user. No way of attaching the disk on an older Windows version afterwards. But that is not the real problem. The real problem is that the upgrade runs as a separate (user-space) process. If this process crashes or your PC crashes or reboots while it runs, your data is gone. There is no feedback how long it still has to run (we've seen multiple days on large volumes) So yeah, maybe avoid ReFS for a few more years...
- temac 3y ago
  Advising ReFS is a little bit insane though. I would certainly not entrust it for my data either.
nyanpasu64 3y ago
One time I accidentally ran a Visual Studio build in a btrfs git clone rather than my main NTFS drive. By the time I noticed and cancelled the build, there were two folders with an identical name but different contents, which I had to delete the folder name twice. I'd say the driver has issues with concurrency.
- Kwpolska 3y ago
  I once ran a `git clone` from WSL1 on the C: drive, and tried to build a C++ project in VS. It complained that "EXAMPLE.H" was not found. An "example.h" file did exist in the repo, and my code asked for "example.h". Turns out WSL1 set some obscure bit not known in Win32 land (but enforced by NTFS) that makes the file names case-sensitive, while VS's path normalisation expects a case-insensitive file system. Perhaps this was related to your issue?
  - nyanpasu64 3y ago
    In a separate occasion, I also got that issue (but worse). I once marked a NTFS folder as case-sensitive to help root out all case mismatch bugs (to get a C++ project eventually building on Linux), but then Visual Studio and CMake started spitting out "file not found" errors even for the correct case! I had somehow produced a "cursed" folder that could not be used for building code until I copied (not moved) its contents to a regular case-insensitive NTFS folder.
summermusic 3y ago
I have run this casually on my main machine for a few years now. I have a Windows partition, a Linux partition (btrfs on LUKS), and a third btrfs partition where I kept my files. I don’t use it often, but when I do I don’t even notice it. It’s as if Windows could just natively read btrfs all along. This was without any “advanced” usage beyond simply accessing, modifying, or deleting files.
Ciantic 3y ago
Heads up, installing both WinBTRFS and OpenZFS on Windows may have problems: "Win OpenZFS driver and WinBtrfs driver dont play well with each other" https://github.com/openzfsonwindows/openzfs/issues/364 https://github.com/openzfsonwindows/openzfs/issues/364

deleted 3y ago

[deleted]

jiggawatts 3y ago

ReFS with Storage Spaces already serves this purpose and is integrated and fully supported. From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong.[1] Using something that is unreliable at best on its native OS shoehorned into Windows is madness. Fun for a lark, sure, but I would never ever entrust this combination with any actual data. [1] “It works for me on my two disk mirror” is an anecdote, not data.

rkagerer 3y ago
Thanks. I tried ReFS when it first came out and it was terribly slow (with data parity on), and Storage Spaces was obscure to set up and manage. Has the landscape improved?
- mgerdts 3y ago
  On WS2022 without patches I noticed that Storage Spaces was only queueing one IO per NVMe device. With current patches queuing is fixed and performance is much better. I think this was fixed sometime in 2023. I’m pretty sure both NTFS and ReFS were affected.
  - jiggawatts 3y ago
    Ah, that would explain the absurdly bad I/O performance I was seeing in Azure VMs that had the new NVMe virtual disk controllers! I had spoken with some of the teams involved and they were rather cagey about the root cause, but at least one person mentioned that there were some fixes in the pipeline for Windows Server 2022 NVMe support. I guess this must have been it!
hsbauauvhabzb 3y ago
> [1] “It works for me on my two disk mirror” is an anecdote, not data. While that statement might well be correct that the quote is in fact an anecdote, the following is also an anecdote: ‘From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong.’
- nisa 3y ago
  Made the mistake using btrfs for a Hadoop cluster at university in kernel 4.x times after reading that SLES uses it and after reading an interview on lwn with someone important, I think the maintainer at that time - that deemed it stable. This must be 10 or 12 years ago or so and it was a wild ride - crashes, manual recovery on 200 machines using clusterssh to get the partitions to mount again. Got out of disk space errors on a 16tb raid 1 (which is not a real raid1) with 5% usage - lot's of sweat I'd rather avoid. Should have just used ext4 in hindsight. For me I decided to not touch it anymore after that experience. I'm sure there is a name for that bias but I don't care. Got burned badly. Lots of people had probably similar experiences and that's were that coming from. Reading the mailing list archives at that time might also be useful to convince yourself that it was more than anecdote.
  - hsbauauvhabzb 3y ago
    I’m not disputing any discourse relating to the factual in/correctness of the anecdote, I’m pointing out that gp is providing an anecdote while disputing anecdotes that they don’t agree with. Provide actual data that’s recent. Linux 4.x was what, 10 years ago? Cars are substantially safer now than they were 10/20/50 years ago, so whose to say your experience with a file system would be different?
    - Haemm0r 3y ago
      Same could be said about cars: Why ever buy a [insert brand] again after you've been burned by its reliabity or other issues? You probably just don't, as the alternatives are good and plenty.
gerdesj 3y ago
One place where ReFS is rather decent is "reflinks" - that's where identical blocks are stored once and in the background the rest are simply links to the one block. That is rather useful in backup systems. XFS also supports reflinks amongst other things and is way older than ReFS and hence considered out of beta (which ReFS isn't, by me) I don't trust data to RefS yet - its a fun project that will no doubt prove itself one day. For now, Windows boxes run NTFS and Linux runs ext4 or XFS.
- Gabrys1 3y ago
  What happens if the very important directory you copied 11 times (just to be sure) ends up producing the same block and doesn't indeed get duplicated as you expected? And now, that block gets corrupted...
  - defrost 3y ago
    Back in the day if I copied (geophysical air) survey data 11 times and put all the copies in the same walk in fire proof safe (in the hanger), that offered no real additional security in the event of a direct hit by an aircraft and explosion while the door was open. If you're going to make 11 copies, they have to go to different physical locations, different devices at least, geographically different places to be sure, or it's pointless. In this instance, block de-duping on a single device makes sense .. expecting mutiple copies on the same device (with or without duplicat block reuse) to offer any additional safety does not.
Dylan16807 3y ago
ReFS only got put back into normal Windows 11 a few months ago. That's a good sign for the future, but it was looking bad for a long time. Also if you turn on data checksums, my understanding is it will delete any file that gets a corrupted sector. And you can only override this behavior on a per-file basis. Unless this changed very recently?
- MarkSweep 3y ago
  Oh, is it no longer exiled to Windows Pro for Workstations? This feature comparison chart still has this there: https://www.microsoft.com/en-us/windows/business/compare-windows-11 https://www.microsoft.com/en-us/windows/business/compare-win... For what it’s worth, regular Windows 10 & 11 Pro (and other editions maybe?) have supported reading and writing ReFS this whole time. It’s just the option to create a new volume that’s been disabled.
  - marwis 3y ago
    It still sort of is but you can create Dev Drive which is based on ReFS
frankjr 3y ago
> From what I’ve heard, BTRFS has a crazy long list of defects where it’ll lock up or corrupt data if you so much as look at it wrong The list cannot be crazy long if Synology uses it for their NASes.
- yjftsjthsd-h 3y ago
  > The list cannot be crazy long if Synology uses it for their NASes. Synology uses a hybrid BTRFS+mdadm arrangement specifically to deal with reliability problems with BTRFS RAID: https://kb.synology.com/en-us/DSM/tutorial/What_was_the_RAID_implementation_for_Btrfs_File_System_on_SynologyNAS https://kb.synology.com/en-us/DSM/tutorial/What_was_the_RAID...
  - wolletd 3y ago
    Which is kind of the point. BTRFS only has issues with RAID5/6 configurations. Using it as a filesystem for a single disk or partition should be totally fine.
    - Dalewyn 3y ago
      Everything I've read about btrfs's RAID5/6 deficiencies is that it can't tolerate sudden losses of power (aka write hole problem), which I think is fine so long as you are aware of it and implement appropriate safety measures such as a UPS or APU. And besides, if you are doing RAID you are probably concerned with the system's uptime which probably means you will have implemented such measures anyway. Note that, yes, I'm aware most home users either aren't aware (nobody RTFM) or are too lazy/cheap to buy a UPS from Office Depot. So perhaps btrfs is warning people to save them from themselves.
      - dark-star 3y ago
        A UPS will not much improve the reliability against sudden power loss. At least here in Europe it is much more likely that a PSU or other component fails than that the power line is suddenly interrupted. And lost writes are a problem that all filesystems have. I recommend reading the paper "Parity Lost and Parity Regained" by Krioukov at USENIX 08...
yjftsjthsd-h 3y ago
Weirdly, it's possible that this version could be more stable/reliable/safe than the Linux version, since it's apparently a wholly independent reimplementation. I suppose it depends on whether BTRFS's problems stem from the underlying data format or the actual code as written for the Linux driver.
dark-star 3y ago
ReFS is terrible. We have seen so many customers lose data on ReFS that I started strongly advising everyone against using it. One example: If you (accidentally or on purpose) attach a ReFS disk or LUN to a newer Windows version, it will be silently upgraded to a new ReFS version without any feedback (or chance to prevent it) for the user. No way of attaching the disk on an older Windows version afterwards. But that is not the real problem. The real problem is that the upgrade runs as a separate (user-space) process. If this process crashes or your PC crashes or reboots while it runs, your data is gone. There is no feedback how long it still has to run (we've seen multiple days on large volumes) So yeah, maybe avoid ReFS for a few more years...
temac 3y ago
Advising ReFS is a little bit insane though. I would certainly not entrust it for my data either.

nyanpasu64 3y ago

One time I accidentally ran a Visual Studio build in a btrfs git clone rather than my main NTFS drive. By the time I noticed and cancelled the build, there were two folders with an identical name but different contents, which I had to delete the folder name twice. I'd say the driver has issues with concurrency.

Kwpolska 3y ago
I once ran a `git clone` from WSL1 on the C: drive, and tried to build a C++ project in VS. It complained that "EXAMPLE.H" was not found. An "example.h" file did exist in the repo, and my code asked for "example.h". Turns out WSL1 set some obscure bit not known in Win32 land (but enforced by NTFS) that makes the file names case-sensitive, while VS's path normalisation expects a case-insensitive file system. Perhaps this was related to your issue?
- nyanpasu64 3y ago
  In a separate occasion, I also got that issue (but worse). I once marked a NTFS folder as case-sensitive to help root out all case mismatch bugs (to get a C++ project eventually building on Linux), but then Visual Studio and CMake started spitting out "file not found" errors even for the correct case! I had somehow produced a "cursed" folder that could not be used for building code until I copied (not moved) its contents to a regular case-insensitive NTFS folder.

summermusic 3y ago

I have run this casually on my main machine for a few years now. I have a Windows partition, a Linux partition (btrfs on LUKS), and a third btrfs partition where I kept my files. I don’t use it often, but when I do I don’t even notice it. It’s as if Windows could just natively read btrfs all along. This was without any “advanced” usage beyond simply accessing, modifying, or deleting files.

Ciantic 3y ago

Heads up, installing both WinBTRFS and OpenZFS on Windows may have problems: "Win OpenZFS driver and WinBtrfs driver dont play well with each other" https://github.com/openzfsonwindows/openzfs/issues/364 https://github.com/openzfsonwindows/openzfs/issues/364