12 ms·

I just don't get it how the Windows world - by far the largest PC platform per userbase - still doesn't have any answer to ZFS. Microsoft had WinFS and then ReF

I just don't get it how the Windows world - by far the largest PC platform per userbase - still doesn't have any answer to ZFS. Microsoft had WinFS and then ReFS but it's on the backburner and while there is active development (Win11 ships some bits time to time) release is nowhere in sight. There are some lone warriors trying the giant task of creating a ZFS compatibility layer with some projects, but they are far from being mature/usable. How come that Windows still uses a 32 year old file system?

mustache_kimono 2y ago
> I just don't get it how the Windows world - by far the largest PC platform per userbase - still doesn't have any answer to ZFS. The mainline Linux kernel doesn't either, and I think the answer is because it's hard and high risk with a return mostly measured in technical respect?
- ffsm8 2y ago
  Technically speaking, bcachefs has been merged into the Linux Kernel - that makes your initial assertion wrong. But considering it's had two drama events within 1 year of getting merged... I think we can safely confirm your conclusion of it being really hard
  - mustache_kimono 2y ago
    > Technically speaking, bcachefs has been merged into the Linux Kernel - that makes your initial assertion wrong. bcachefs doesn't implement its erasure coding/RAID yet? Doesn't implement send/receive. Doesn't implement scrub/fsck. See: https://bcachefs.org/Roadmap https://bcachefs.org/Roadmap, https://bcachefs.org/Wishlist/ https://bcachefs.org/Wishlist/ btrfs is still more of a legit competitor to ZFS these days and it isn't close to touching ZFS where it matters. If the perpetually half-finished bcachefs and btrfs are the "answer" to ZFS that seems like too little, too late to me.
    - deleted 2y ago
      [deleted]
    - koverstreet 2y ago
      Erasure coding is almost done; all that's missing is some of the device evacuate and reconstruct paths, and people have been testing it and giving positive feedback (especially w.r.t. performance). It most definitely does have fsck and has since the beginning, and it's a much more robust and dependable fsck than btrfs's. Scrub isn't quite done - I actually was going to have it ready for this upcoming merge window except for a nasty bout of salmonella :) Send/recv is a long ways off, there might be some low level database improvements needed before that lands. Short term (next year or two) priorities are finishing off online fsck, more scalability work (upcoming version for this merge window will do 50PB, but now we need to up the limit on number of drives), and quashing bugs.
      - ryao 2y ago
        Hearing that it is missing some code for reconstruction makes it sound like it is missing something fairly important. The original purpose of parity RAID is to support reconstruction.
        koverstreet 2y ago
        We can do reconstruct reads, what's missing is the code to rewrite missing blocks in a stripe after a drive dies. In general, due to the scope of the project, I've been prioritizing the functionality that's needed to validate the design and the parts that are needed for getting the relationships between different components correct. e.g. recently I've been doing a bunch of work on backpointers scalability, and that plus scrub are leading to more back and forth iteration on minor interactions with erasure coding. So: erasure coding is complete enough to know that it works and for people to torture test it, but yes you shouldn't be running it in production yet (and it's explicitly marked as such). What's remaining is trivial but slightly tedious stuff that's outside the critical path of the rest of the design. Some of the code I've been writing for scrub is turning out to also be what we want for reconstruct, so maybe we'll get there sooner rather than later...
kwanbix 2y ago
Honest question. As an end user that uses Windows and Linux and does not uses ZFS, what I am missing?
- madeofpalk 2y ago
  I'm missing file clones/copy-on-write.
- poisonborz 2y ago
  Way better data security, resilience against file rotting. This goes for both HDDs or SSDs. Copy-on-write, snapshots, end to end integrity. Also easier to extend the storage for safety/drive failure (and SSDs corrupt in a more sneaky way) with pools.
  - wil421 2y ago
    How many of us are using single disks on our laptops? I have a NAS and use all of the above but that doesn’t help people with single drive systems. Or help me understand why I would want it on my laptop.
    - ekianjo 2y ago
      It provides encryption by default without having to deal with LUKS. And no need to ever do fsck again.
      - Twey 2y ago
        Except that swap on OpenZFS still deadlocks 7 years later (https://github.com/openzfs/zfs/issues/7734 https://github.com/openzfs/zfs/issues/7734) so you're still going to need LUKS for your swap anyway.
        ryao 2y ago
        Another option is to go without swap. I avoid swap on my machines unless I want hibernation support.
badgersnake 2y ago
NTFS is good enough for most people, who have a laptop with one SSD in it.
- wkat4242 2y ago
  The benefits of ZFS don't need multiple drives to be useful. I'm running ZFS on root for years now and snapshots have saved my bacon several times. Also with block checksums you can at least detect bitrot. And COW is always useful.
  - zamadatix 2y ago
    Windows manages volume snapshots on NTFS through VSS. I think ZFS snapshots are a bit "cleaner" of a design, and the tooling is a bit friendlier IMO, but the functionality to snapshot, rollback, and save your bacon is there regardless. Outside of the automatically enabled "System Restore" (which only uses VSS to snapshot specific system files during updates) I don't think anyone bothers to use it though. CoW, advanced parity, and checksumming are the big ones NTFS lacks. CoW is just inherently not how NTFS is designed and checksumming isn't there. Anything else (encryption, compression, snapshots, ACLs, large scale, virtual devices, basic parity) is done through NTFS on Windows.
    - wkat4242 2y ago
      Yes I know that NTFS has snapshots, I mentioned that in another comment. I don't think NTFS is as relevant in comparison though. People who choose windows will have no interest in ZFS and vice versa (someone considering ZFS will not pick Windows). And I don't think anyone bothers to use it due to the lack of user-facing tooling around it. If it would be as easy to create snapshots as it is on ZFS, more people would use it, I'm sure. It's just so amazing to try something out, screw up my system and just revert :P But VSS is more of a system API than a user-facing geature. VSS is also used by backup software to quiet the filesystem by the way. But yeah the others are great features. My main point was though that almost all the features of ZFS are very beneficial even on a single drive. You don't need an array to take advantage of Snapshots, the crash reliability that CoW offers, and checksumming (though you will lack the repair option obviously)
GuB-42 2y ago
To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making and still doesn't match ZFS in features and stability. Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. In all fairness, there are few OS components that are as critical as the filesystem, and many wouldn't touch filesystems that have less than a decade of proven track record in production.
- lousken 2y ago
  as far as stability goes, btrfs is used by meta, synology and many others, so I wouldn't say it's not stable, but some features are lacking
  - fourfour3 2y ago
    Do Synology actually use the multi-device options of btrfs, or are they using linux softraid + lvm underneath? I know Synology Hybrid RAID is a clever use of LVM + MD raid, for example.
    - phs2501 2y ago
      I believe Synology runs btrfs on top of regular mdraid + lvm, possibly with patches to let btrfs checksum failures reach into the underlying layers to find the right data to recover. Related blog post: https://daltondur.st/syno_btrfs_1/ https://daltondur.st/syno_btrfs_1/
      - fourfour3 2y ago
        That was very interesting reading, thanks!
  - azalemeth 2y ago
    My understanding is that single-disk btrfs is good, but raid is decidedly dodgy; https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid56-status-and-recommended-practices https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid5... states that: > The RAID56 feature provides striping and parity over several devices, same as the traditional RAID5/6. > There are some implementation and design deficiencies that make it unreliable for some corner cases and *the feature should not be used in production, only for evaluation or testing*. > The power failure safety for metadata with RAID56 is not 100%. I have personally been bitten once (about 10 years ago) by btrfs just failing horribly on a single desktop drive. I've used either mdadm + ext4 (for /) or zfs (for large /data mounts) ever since. Zfs is fantastic and I genuinely don't understand why it's not used more widely.
ryao 2y ago
What do you mean by a ZFS compatibility layer? There is a Windows port: https://github.com/openzfsonwindows/openzfs https://github.com/openzfsonwindows/openzfs Note that it is a beta.
MauritsVB 2y ago
There is occasional talk of moving the Windows implementation of OpenZFS (https://github.com/openzfsonwindows/openzfs/releases https://github.com/openzfsonwindows/openzfs/releases) into an officially supported tier, though that will probably come after the MacOS version (https://github.com/openzfsonosx https://github.com/openzfsonosx) is officially supported.
bayindirh 2y ago
> How come that Windows still uses a 32 year old file system? Simple. Because most of the burden is taken by the (enterprise) storage hardware hosting the FS. Snapshots, block level deduplication, object storage technologies, RAID/Resiliency, size changes, you name it. Modern storage appliances are black magic, and you don't need much more features from NTFS. You either transparently access via NAS/SAN or store your NTFS volumes on capable disk boxes. On the Linux world, at the higher end, there's Lustre and GPFS. ZFS is mostly for resilient, but not performance critical needs.
- poisonborz 2y ago
  So private consumers should just pay cloud subscription if they want safer/modern data storage for their PC? (without NAS)
  - bayindirh 2y ago
    I think Microsoft has discontinued Windows 7 backup to force people to buy OneDrive subscriptions. They also forcefully enabled the feature when they first introduced it. So, I think that your answer for this question is "unfortunately, yes". Not that I support the situation.
  - BSDobelix 2y ago
    If you need Windows, you can use something like restic (checksums and compression) and external drives (more than one, stored in more than one place) to make a backup. Plus "maybe" but not needed ReFS (on your non-Windows partition), which is included in the Workstation/Enterprise editions of Windows. I trust my own backups much more than any subscription, not essentially from a technical point of view, but from an access point of view (e.g. losing access to your Google account). EDIT: You have to enable check-summing and/or compression for data on ReFS manually https://learn.microsoft.com/en-us/windows-server/storage/refs/refs-overview https://learn.microsoft.com/en-us/windows-server/storage/ref...
    - bayindirh 2y ago
      > I trust my own backups much more than any subscription, not from a technical standpoint but from an access one (for example, losing access to your google account). I personally use cloud storage extensively, but I keep a local version with periodic rclone/borg. It allows me access from everywhere and sleep well at night.
zamadatix 2y ago
NTFS was able to be extended in various way over the years to the point what you could do with an NTFS drive 32 years ago will feel like talking about a completely different filesystem than what you can do with it on current Windows. Honestly I really like ReFS, particularly in context of storage spaces, but I don't think it's relevant to Microsoft's consumer desktop OS where users don't have 6 drives they need to pool together. Don't get me wrong, I use ZFS because that's what I can get running on a Linux server and I'm not going to go run Windows Server just for the storage pooling... but ReFS + Storage Spaces wins my heart with the 256 MB slab approach. This means you can add+remove mixed sized drives and get the maximum space utilization for the parity settings of the pool. Here ZFS is still getting to online adds of same or larger drives 10 years later.
nickdothutton 2y ago
OS development pretty much stopped around 2000. ZFS is from 2001. I don't count a new way to organise my photos or integrate with a search engine as "OS" though.
doctorpangloss 2y ago
The same reason file deduplication is not enabled for client Windows: greed. For example, there are numerous new file systems people use: OneDrive, Google Drive, iCloud Storage. Do you get it?
_gea 2y ago
OpenZFS on Windows has reached release candidate state (rc11) with some remaining bugs around volume mount (nothing serious, avoid encryption atm). Development is very fast (a new rc every few weeks to fix remaining problems) Paired with Storage Spaces to pool disks of any type or size with location, redundancy or tiering defined per Space, Windows is now a premium storage platform. With a Windows Server ex a cheap 2022/2025 Essentials you additionally can use SMB Direct/RDMA with a performance up to 10 Gbyte/s over lan and virtual harddisks .vhdx as a zero config and faster alternative to iSCSI. This is why I have ported me ZFS web-gui napp-it cs from Solaris to Windows to manage Storage Spaces and ZFS and remotely OpenZFS servers like Proxmox (free for noncommercial use)

mustache_kimono 2y ago

> I just don't get it how the Windows world - by far the largest PC platform per userbase - still doesn't have any answer to ZFS. The mainline Linux kernel doesn't either, and I think the answer is because it's hard and high risk with a return mostly measured in technical respect?

ffsm8 2y ago
Technically speaking, bcachefs has been merged into the Linux Kernel - that makes your initial assertion wrong. But considering it's had two drama events within 1 year of getting merged... I think we can safely confirm your conclusion of it being really hard
- mustache_kimono 2y ago
  > Technically speaking, bcachefs has been merged into the Linux Kernel - that makes your initial assertion wrong. bcachefs doesn't implement its erasure coding/RAID yet? Doesn't implement send/receive. Doesn't implement scrub/fsck. See: https://bcachefs.org/Roadmap https://bcachefs.org/Roadmap, https://bcachefs.org/Wishlist/ https://bcachefs.org/Wishlist/ btrfs is still more of a legit competitor to ZFS these days and it isn't close to touching ZFS where it matters. If the perpetually half-finished bcachefs and btrfs are the "answer" to ZFS that seems like too little, too late to me.
  - deleted 2y ago
    [deleted]
  - koverstreet 2y ago
    Erasure coding is almost done; all that's missing is some of the device evacuate and reconstruct paths, and people have been testing it and giving positive feedback (especially w.r.t. performance). It most definitely does have fsck and has since the beginning, and it's a much more robust and dependable fsck than btrfs's. Scrub isn't quite done - I actually was going to have it ready for this upcoming merge window except for a nasty bout of salmonella :) Send/recv is a long ways off, there might be some low level database improvements needed before that lands. Short term (next year or two) priorities are finishing off online fsck, more scalability work (upcoming version for this merge window will do 50PB, but now we need to up the limit on number of drives), and quashing bugs.
    - ryao 2y ago
      Hearing that it is missing some code for reconstruction makes it sound like it is missing something fairly important. The original purpose of parity RAID is to support reconstruction.
      - koverstreet 2y ago
        We can do reconstruct reads, what's missing is the code to rewrite missing blocks in a stripe after a drive dies. In general, due to the scope of the project, I've been prioritizing the functionality that's needed to validate the design and the parts that are needed for getting the relationships between different components correct. e.g. recently I've been doing a bunch of work on backpointers scalability, and that plus scrub are leading to more back and forth iteration on minor interactions with erasure coding. So: erasure coding is complete enough to know that it works and for people to torture test it, but yes you shouldn't be running it in production yet (and it's explicitly marked as such). What's remaining is trivial but slightly tedious stuff that's outside the critical path of the rest of the design. Some of the code I've been writing for scrub is turning out to also be what we want for reconstruct, so maybe we'll get there sooner rather than later...

kwanbix 2y ago

Honest question. As an end user that uses Windows and Linux and does not uses ZFS, what I am missing?

madeofpalk 2y ago
I'm missing file clones/copy-on-write.
poisonborz 2y ago
Way better data security, resilience against file rotting. This goes for both HDDs or SSDs. Copy-on-write, snapshots, end to end integrity. Also easier to extend the storage for safety/drive failure (and SSDs corrupt in a more sneaky way) with pools.
- wil421 2y ago
  How many of us are using single disks on our laptops? I have a NAS and use all of the above but that doesn’t help people with single drive systems. Or help me understand why I would want it on my laptop.
  - ekianjo 2y ago
    It provides encryption by default without having to deal with LUKS. And no need to ever do fsck again.
    - Twey 2y ago
      Except that swap on OpenZFS still deadlocks 7 years later (https://github.com/openzfs/zfs/issues/7734 https://github.com/openzfs/zfs/issues/7734) so you're still going to need LUKS for your swap anyway.
      - ryao 2y ago
        Another option is to go without swap. I avoid swap on my machines unless I want hibernation support.

badgersnake 2y ago

NTFS is good enough for most people, who have a laptop with one SSD in it.

wkat4242 2y ago
The benefits of ZFS don't need multiple drives to be useful. I'm running ZFS on root for years now and snapshots have saved my bacon several times. Also with block checksums you can at least detect bitrot. And COW is always useful.
- zamadatix 2y ago
  Windows manages volume snapshots on NTFS through VSS. I think ZFS snapshots are a bit "cleaner" of a design, and the tooling is a bit friendlier IMO, but the functionality to snapshot, rollback, and save your bacon is there regardless. Outside of the automatically enabled "System Restore" (which only uses VSS to snapshot specific system files during updates) I don't think anyone bothers to use it though. CoW, advanced parity, and checksumming are the big ones NTFS lacks. CoW is just inherently not how NTFS is designed and checksumming isn't there. Anything else (encryption, compression, snapshots, ACLs, large scale, virtual devices, basic parity) is done through NTFS on Windows.
  - wkat4242 2y ago
    Yes I know that NTFS has snapshots, I mentioned that in another comment. I don't think NTFS is as relevant in comparison though. People who choose windows will have no interest in ZFS and vice versa (someone considering ZFS will not pick Windows). And I don't think anyone bothers to use it due to the lack of user-facing tooling around it. If it would be as easy to create snapshots as it is on ZFS, more people would use it, I'm sure. It's just so amazing to try something out, screw up my system and just revert :P But VSS is more of a system API than a user-facing geature. VSS is also used by backup software to quiet the filesystem by the way. But yeah the others are great features. My main point was though that almost all the features of ZFS are very beneficial even on a single drive. You don't need an array to take advantage of Snapshots, the crash reliability that CoW offers, and checksumming (though you will lack the repair option obviously)

GuB-42 2y ago

To be honest, the situation with Linux is barely better. ZFS has license issues with Linux, preventing full integration, and Btrfs is 15 years in the making and still doesn't match ZFS in features and stability. Most Linux distros still use ext4 by default, which is 19 years old, but ext4 is little more than a series of extensions on top of ext2, which is the same age as NTFS. In all fairness, there are few OS components that are as critical as the filesystem, and many wouldn't touch filesystems that have less than a decade of proven track record in production.

lousken 2y ago
as far as stability goes, btrfs is used by meta, synology and many others, so I wouldn't say it's not stable, but some features are lacking
- fourfour3 2y ago
  Do Synology actually use the multi-device options of btrfs, or are they using linux softraid + lvm underneath? I know Synology Hybrid RAID is a clever use of LVM + MD raid, for example.
  - phs2501 2y ago
    I believe Synology runs btrfs on top of regular mdraid + lvm, possibly with patches to let btrfs checksum failures reach into the underlying layers to find the right data to recover. Related blog post: https://daltondur.st/syno_btrfs_1/ https://daltondur.st/syno_btrfs_1/
    - fourfour3 2y ago
      That was very interesting reading, thanks!
- azalemeth 2y ago
  My understanding is that single-disk btrfs is good, but raid is decidedly dodgy; https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid56-status-and-recommended-practices https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid5... states that: > The RAID56 feature provides striping and parity over several devices, same as the traditional RAID5/6. > There are some implementation and design deficiencies that make it unreliable for some corner cases and *the feature should not be used in production, only for evaluation or testing*. > The power failure safety for metadata with RAID56 is not 100%. I have personally been bitten once (about 10 years ago) by btrfs just failing horribly on a single desktop drive. I've used either mdadm + ext4 (for /) or zfs (for large /data mounts) ever since. Zfs is fantastic and I genuinely don't understand why it's not used more widely.

ryao 2y ago

What do you mean by a ZFS compatibility layer? There is a Windows port: https://github.com/openzfsonwindows/openzfs https://github.com/openzfsonwindows/openzfs Note that it is a beta.

MauritsVB 2y ago

There is occasional talk of moving the Windows implementation of OpenZFS (https://github.com/openzfsonwindows/openzfs/releases https://github.com/openzfsonwindows/openzfs/releases) into an officially supported tier, though that will probably come after the MacOS version (https://github.com/openzfsonosx https://github.com/openzfsonosx) is officially supported.

bayindirh 2y ago

> How come that Windows still uses a 32 year old file system? Simple. Because most of the burden is taken by the (enterprise) storage hardware hosting the FS. Snapshots, block level deduplication, object storage technologies, RAID/Resiliency, size changes, you name it. Modern storage appliances are black magic, and you don't need much more features from NTFS. You either transparently access via NAS/SAN or store your NTFS volumes on capable disk boxes. On the Linux world, at the higher end, there's Lustre and GPFS. ZFS is mostly for resilient, but not performance critical needs.

poisonborz 2y ago
So private consumers should just pay cloud subscription if they want safer/modern data storage for their PC? (without NAS)
- bayindirh 2y ago
  I think Microsoft has discontinued Windows 7 backup to force people to buy OneDrive subscriptions. They also forcefully enabled the feature when they first introduced it. So, I think that your answer for this question is "unfortunately, yes". Not that I support the situation.
- BSDobelix 2y ago
  If you need Windows, you can use something like restic (checksums and compression) and external drives (more than one, stored in more than one place) to make a backup. Plus "maybe" but not needed ReFS (on your non-Windows partition), which is included in the Workstation/Enterprise editions of Windows. I trust my own backups much more than any subscription, not essentially from a technical point of view, but from an access point of view (e.g. losing access to your Google account). EDIT: You have to enable check-summing and/or compression for data on ReFS manually https://learn.microsoft.com/en-us/windows-server/storage/refs/refs-overview https://learn.microsoft.com/en-us/windows-server/storage/ref...
  - bayindirh 2y ago
    > I trust my own backups much more than any subscription, not from a technical standpoint but from an access one (for example, losing access to your google account). I personally use cloud storage extensively, but I keep a local version with periodic rclone/borg. It allows me access from everywhere and sleep well at night.

zamadatix 2y ago

NTFS was able to be extended in various way over the years to the point what you could do with an NTFS drive 32 years ago will feel like talking about a completely different filesystem than what you can do with it on current Windows. Honestly I really like ReFS, particularly in context of storage spaces, but I don't think it's relevant to Microsoft's consumer desktop OS where users don't have 6 drives they need to pool together. Don't get me wrong, I use ZFS because that's what I can get running on a Linux server and I'm not going to go run Windows Server just for the storage pooling... but ReFS + Storage Spaces wins my heart with the 256 MB slab approach. This means you can add+remove mixed sized drives and get the maximum space utilization for the parity settings of the pool. Here ZFS is still getting to online adds of same or larger drives 10 years later.

nickdothutton 2y ago

OS development pretty much stopped around 2000. ZFS is from 2001. I don't count a new way to organise my photos or integrate with a search engine as "OS" though.

doctorpangloss 2y ago

The same reason file deduplication is not enabled for client Windows: greed. For example, there are numerous new file systems people use: OneDrive, Google Drive, iCloud Storage. Do you get it?

_gea 2y ago

OpenZFS on Windows has reached release candidate state (rc11) with some remaining bugs around volume mount (nothing serious, avoid encryption atm). Development is very fast (a new rc every few weeks to fix remaining problems) Paired with Storage Spaces to pool disks of any type or size with location, redundancy or tiering defined per Space, Windows is now a premium storage platform. With a Windows Server ex a cheap 2022/2025 Essentials you additionally can use SMB Direct/RDMA with a performance up to 10 Gbyte/s over lan and virtual harddisks .vhdx as a zero config and faster alternative to iSCSI. This is why I have ported me ZFS web-gui napp-it cs from Solaris to Windows to manage Storage Spaces and ZFS and remotely OpenZFS servers like Proxmox (free for noncommercial use)