12 ms·
WinBtrfs – A Windows driver for the next-generation Linux filesystem Btrfs
- lordkrandel 9y agoWasn't BTRFS deprecated? https://www.phoronix.com/scan.php?page=news_item&px=Red-Hat-Deprecates-Btrfs-Again https://www.phoronix.com/scan.php?page=news_item&px=Red-Hat-...
- usefulcat 9y agoThat just means that RedHat is not interested in offering support for it in the future.
- sebazzz 9y agoLooks interesting. If this is possible, I wonder why a driver for a much more common filesystem like ext hasn't picked of (e2fs driver does not count, that is ext2 only). Why do you also check the artifacts in?
- metafex 9y agoThere is ext2fsd (https://sourceforge.net/projects/ext2fsd https://sourceforge.net/projects/ext2fsd). Worked like a charm last time I had to use it. Last version is even from July. edit: Unsupported features: 1, journal: log-based operations, external journal 2, EA (extended attributes), ACL support
- seiferteric 9y agoYa I am using this and works with ext3/4 as well, but as you mentioned, there are some unsupported features. Unfortunately that means that it messes with some metadata or something and I have to run e2fsck when I reboot into linux. You can read about it here https://askubuntu.com/questions/849872/how-can-i-prevent-windows-10-from-corrupting-the-ext4-superblock-every-time https://askubuntu.com/questions/849872/how-can-i-prevent-win...
- deleted 9y ago[deleted]
- newscracker 9y agoThis seems like a good start, and it could help provide more choice in cross-platform data storage and data exchange. Recently I looked once again, as I usually do once every few years, at options for exchanging data through external drives between Windows, macOS and Linux (I prefer USB drives, which are faster than speeds I can attain over the network). Shockingly, it seemed to me like NTFS was the easier filesystem to go with (Linux has good support for it, and macOS needs a third party driver for write support)! Back to this project, I feel there is a strong need to have a set of GUI tools (beyond Windows Shell Extensions) to take advantage of the btrfs filesystem features (like creating snapshots, sending snapshots to backup storage, etc.). I use btrfs on Linux, and the lack of good, up-to-date GUI tools to use its feature set has been a source of frustration (yes, I can use the command line, but then that would restrict the usage only to me, leaving out the other not-so-tech-savvy people around).
- teddyh 9y ago> options for exchanging data through external drives between Windows, macOS and Linux How is the UDF support on macOS? I ask since I relatively recently chose UDF as the Windows/Linux common HDD file system format, but I did not consider macOS.
- cmurf 9y agoThe trick for cross platform UDF is formatting the whole stick as UDF, without partitioning (no MBR, GPT, or APM). In that case, it works, I tested this just last week on Windows 10, Fedora 26, macOS 10.12. It's totally sane. It's "like" FAT32 except you don't have the 2G filesize limitation. NTFS might also be sane, but out of the box macOS doesn't have write support.
- jaclaz 9y agoAFAIK it is fully supported, I also use often UDF, though for whatever reasons there are not so many users around, after all it is a formally documented and "standard" filesystem ISO/IEC 13346 and ECMA-167: https://en.wikipedia.org/wiki/Universal_Disk_Format https://en.wikipedia.org/wiki/Universal_Disk_Format but somehow everyone thinks that it is only fod CD/DVD's.
- ehutch79 9y agoI thought btrfs was being phased out in linux distros?
- jlgaddis 9y agoJust RHEL. Certainly not OpenSUSE.
- rantanplan 9y agoJust RHEL. The biggest(or 2nd biggest) contributor to the Linux kernel.
- kuschku 9y agoAnd smallest contributor to btrfs
- akerro 9y agoIt was nothing more than a political move.
- georgyo 9y agoThe smallest btrfs contributer, redhat, phased out support. They were the only ones to do so.
- sabujp 9y agogreat, so when do we get xfs natively in windows?
- oneplane 9y agoProbably never. We don't even have ext3 or ext4 support, only some bare ext2. And then there is the dozen or so other filesystems that would be handy to have. Microsoft doesn't care, they also don't have a business case for it, and they rather reinvent the wheel instead, using their own fs. The FOSS community doesn't really care that much either, mostly because Linux can deal with NTFS, and the other way around happens far less, so no real push has been made for it.
- joshstrange 9y agoI'm using BTRFS on my SSD and while it has some neat features the need to rebalance periodically or you can "run out of space" while using less than half the SSD drives me crazy.
- cmurf 9y agoKernel 4.8 and newer have a new ticketed enospc infrastructure that should avoid this, there's also earlier changes where empty block groups are deallocated automatically. If you're already using 4.8+ then try 4.12.6 or newer, which includes commit 4a309747. Due to past long standing enospc challenges, more experienced users are in the (bad) habit of micro-managing Btrfs manually with filtered balance, e.g. -dusage=15 -musage=15. Really we kinda need to stop doing that to find and fix the remaining edge cases; or worst case upstream should deploy a systemd timer / cron job to do these partial balances maybe once a week. Anyway I don't balance at all on my drives and haven't run into enospc in quite a while. So chances are you're hitting an edge case, and there's a bug that needs to be tracked down if you're already using 4.12.6+.
- joshstrange 9y agoWhen I get home I'll have to check. I'm using it via UnRAID so I don't have as much control over the version I run but they are pretty good about releasing updates, I'm just bad about applying them and restarting because taking down Plex is a sin punishable by death lol
- cmurf 9y agoLooks like that same 4.12.6 fix is in 4.9.42.
- cmurf 9y agoWould this be easier, or harder if it were FUSE based?
- Xophmeister 9y agoFUSE is a Unix thing. I believe there is a port/similar thing for Windows, but I guess it would suffer the same kinds of limitations; namely performance, because everything is translated through userspace. That said, isn't it true that Windows drivers are userspace-based, anyway?
- dom0 9y agoThere are multiple projects providing user-space file systems for Windows, dokany and WinFSP for example. > That said, isn't it true that Windows drivers are userspace-based, anyway? No.
- Xophmeister 9y agoFair enough :)
- invokestatic 9y agoAll file system drivers must be kernel-mode on Windows. While Windows does have usermode drivers (UMDF), it's significantly gimped. Mostly just simple USB drivers or WDDM display drivers. There have been a few projects to try to bring FUSE-like functionality to Windows like Dokan but I believe these are still too immature.
- icebraining 9y agoWinFsp seems pretty good: https://github.com/billziss-gh/winfsp https://github.com/billziss-gh/winfsp
- mdip 9y agoThere is a "sort-of" port (you were correct in saying a "similar thing for Windows"). It's Dokany: https://github.com/dokan-dev/dokany https://github.com/dokan-dev/dokany Keybase.io and sshfs use it, currently. Personally, I love it. I'm sure you're right about performance limitations, but there are very good use cases for it, like Keybase Filesystem and sshfs, where any latency introduced by the driver subsystem is less likely to be felt due to the latency of working with files over the network. Interestingly, a set of the insider previews of Windows broke Dokany so badly that launching the driver caused everything to hang (hard -- all but the mouse stopped responding to anything, even CTRL+ALT+DELETE). It's tricky stuff, apparently.
- pjc50 9y agoNice work. I've starred this simply because examples of Windows filesystem driver source are extremely rare. As are people familiar with both Windows and Linux development, which have very differnt styles and conventions.
- alfredxing 9y agoWhat's interesting to me is that there doesn't seem to be a modern filesystem that works across all major platforms. btrfs has Linux and this (unofficial) Windows driver; ZFS has Linux and OS X support but nothing for Windows. NTFS is pretty much Windows and Linux only for good write access, and APFS is macOS-only. I realize that designing a filesystem and writing drivers for it are very difficult tasks, but surely it would be extremely beneficial to have at least 1 open source filesystem that has good support across all platforms?
- kasabali 9y agoUDF https://en.m.wikipedia.org/wiki/Universal_Disk_Format https://en.m.wikipedia.org/wiki/Universal_Disk_Format
- marcoperaza 9y agoI believe that part of the issue is that the file systems are tailored to the needs of the operating system they run on. Take file permissions. As the basic permissions model, Linux uses user/group/everyone, while Windows uses ACLs. Linux also supports ACLs but they don't work in the same way that Windows' does. NTFS is built to meet the needs of Windows, while Ext4 is built to meet the needs of Linux. You can use each on the other OS with the right drivers, but just as data drives with very roughly mapped permissions.
- alfredxing 9y agoHmm, that's a good point! However I don't think this would be a major hurdle: file permissions offer OS-level security only, not disk-level security/encryption, so it would still make sense for different OS drivers to implement only their specific permissions attributes.
- thomastjeffery 9y agoTo make your point more clear: NTFS is built for Windows, while every other modern filesystem is made for OS's with Unix-style permissions (including BSDs like OS X).
- _benj 9y agoThis should have been done by Microsoft long ago, even before their little "Microsoft <3 Linux" gimmick with bash. Still, a step in the right direction, at least on my book. My shared drives are all exFAT and that seems to work fine out of the box for Linux, Win and macOS.
- mjevans 9y agoThat and ext4 support (at the very least).
- jsiepkes 9y agoReally cool! Does anyone know of the other way around; An open source Windows next gen FS (ReFS) driver for Linux?
- mdip 9y agoI suspect someone is already playing the idea of implementing support for it, but if NTFS is any indication, it'll take a long time before anything stable exists. There's also less motivation for implementing this filesystem in Linux at this point. It's available only on Server and Windows 10 Workstation (at least for creating -- I believe Win10 is able to read/work with ReFS in any version). It's not feature-complete yet[0], missing some important ones like deduplication. It's quite new, so there isn't a lot of it in the wild, reducing the need to make a compatible implementation for interoperability (probably the #1 reason for making a compatible version; #2 being to have a cool new filesystem with features that don't exist in currently available offerings). And being proprietary, it will almost certainly be a very difficult thing to write -- much like NTFS is/was. [0] Not a show stopper - NTFS evolved over time, as all filesystems seem to.
- rootw0rm 9y agoReFS isn't even complete yet. I'd never trust it, either. Microsoft has zero tools for fixing shit when their buggy filesystems break. Had a Server 2K12 software RAID5 once upon a time, obviously a big mistake for many reasons...
- brian_herman 9y agoWhy is it a reimplementation from scratch? Why not use the linux kernel source?
- awalton 9y agoWhy not reimplement it and prove the filesystem can be reimplemented sanely, and furthermore now has multiple implementations to test the specification against?
- mdip 9y agoI'd wondered this, too. My first (wrong) assumption was that the author was aiming for a different license with the Windows version, but it's GPL. If I had a guess, aside from "because I wanted to see if I could", it might be because of inherent differences in the way the OSes behave with regard to filesystems[0] would make a re-implementation from spec work better (faster, more reliably or whatever), though the author goes quite a ways to point out that he's used none of the original source, as if to state it's a clean-room implementation and I'd speculate (wildly) that though there are likely more than a few differences in Linux/Windows' APIs, there is almost certainly a lot about how btrfs works that isn't different when implemented on both platforms. These APIs would affect how the code is designed, and attempting to port by converting API calls that are Linux specific and plugging in similar Windows API calls could lead to a less-ideal implementation than if one read the spec and designed it targeting the operating system's API directly[1]. And then there's my original, wrong, assumption: if this individual is the only author, and it appears they might be, it does give him/her the ability to dual-license it later; i.e. offer a commercial license which doesn't have GPL-related requirements or switch it out to a less restrictive Open Source license if the author so chooses. [0] I'm not an expert or even a n00b in this area. I, almost completely, know nothing about filesystem API differences between Linux and Windows since I don't know either the kernel interfaces for Linux or the Windows APIs related to the filesystem at any level having never written anything targeting that technology, directly. [1] A former project that I built a long time ago comes to mind where I tasked with porting a tool to change what it used for storing records. The technology I started with had no concept of saving array records to disk, requiring a separate write for each record. Non-ideal, but not terrible because the technology was optimised at doing many small writes. The new target could accept an array of records and write them all in one call. I did a "compatible" re-implementation, replacing calls with their equivalent in the new record store and the performance made the product unusable. It wasn't until I redid all of the record I/O with the target platform in mind that things started working properly.
- rkeene2 9y agoOh the humanity. As someone switching away from BtrFS everywhere, everytime I've tried it over the years, I can just imagine what a Windows port "feels" like.
- jhack 9y agoAnyone know if there are any implementations for OSX/macOS? I tried OpenZFS and that worked great (tested on an SD card with LZ4 compression) but I didn't like the memory pressure it was putting on my system, even after tweaking. I really wish APFS supported transparent compression.
- gerdesj 9y ago"What's interesting to me is that there doesn't seem to be a modern filesystem that works across all major platforms." What's interesting to me is that there is no modern filesystem that has a useful ACL mechanism. Say you have a tree of say 100M files and subdirs: now give rights at various points - it takes ages. Your complaint can be worked around by remounting via an intermediary system and VMs are cheap. Mine is not really a complaint - per se - but a challenge: Why on earth can NSS (off of the 90s) do rich, recursive perms based on a "trustee assignment" on the fly but NTFS and Unix's finest not?