6 ms·

WinFsp · Windows File System Proxy

151 points by mariuz 3y ago

Dwedit 3y ago
I wonder how this compares to Dokan. Dokan doesn't integrate into the Windows Cache Manager, and file performance is constrained vs reading something which was recently read.
- jarym 3y ago
  I’d like to know this too - Dokan wasn’t the most stable when I used it years ago (maybe it’s improved since then), would love to know how this is different
  - Dwedit 3y ago
    I tried out Dokan a couple years ago, and it seemed decently stable, just underperforming. There was a big fix that tremendously increased performance and also fixed a race condition around that time. On a completely different note, there's also "Windows Projected File System", which sounds a lot like a user-mode file system, but it's not. It requires that files be backed by real files on your disk.
    - themerone 3y ago
      By my reading of the docs, Windows Projected File System (ProjFS) only requires reliability and performance characteristics comparable to a local filesystem. I think it would work if your data is in memory or backed by a 10G+ lan.
    - accoil 3y ago
      Yeah, that was a surprise to me when I toyed with Windows Projected File System. Turned off the program, and all my "pretend" files were still there.
ehPReth 3y ago
rclone uses this! it’s great!
- noisem4ker 3y ago
  Another neat application is SSHFS-Win, from the same authors of WinFsp itself. https://github.com/winfsp/sshfs-win https://github.com/winfsp/sshfs-win
MuffinFlavored 3y ago
This is super cool. Why would somebody need to use this instead of like, smb or however normal Windows machines on the same network share drives/files/folders (even with other Mac machines on the network, or Linux, etc.)?
- Dwedit 3y ago
  This isn't for network shares, this is for reading/writing programmatically generated files. The possibilities are endless. For example, you could make a "filesystem" where everything inside looks like an uncompressed BMP file, but the actual files backing them are some other image format. Then ancient software would be able to use the newest image formats.
- themerone 3y ago
  That is a reasonable approach, one of the 2 MacOS fuse implementations uses NFS. I've never written an SMB server, but I assume it is a lot more complicated than using WinFsp or Fuse. WSL 2 uses the 9P network filesystem to share files between Linux & Windows. Supposedly this is a simple protocol to implement. I've searched, but can't find any documentation on using 9P without WSL.
  - hrunt 3y ago
    Cryptomator suggests WinFSP for its Windows installation, MacFUSE for Intel Macs, and FUSE-T (the NFS FUSE implementation) for Apple silicon Macs. I've used all of them, and I've also used Dokan with encfs. Both OSes treat network filesystems differently from local disks. The FUSE-T implementation is very speedy, but it shows up in the finder as a connection to "localhost", not as a properly named mount. In addition, macOS creates .DS_Store files on network drives by default (there is a setting you can change from the CLI to prevent this). On Windows, the filesystems can be mounted as local drives or network drives. When mounted as network drives, they can have issues with indexing and trust. All-in-all, drives you expect to function as local drives work best when they're actually local device filesystems and not network mounts.
- ww520 3y ago
  This is for exposing other data as a virtual file system. Let's say you want to expose the Git versions as folders and files. You need this to write the file system layer on top of Git.
nashashmi 3y ago
Database Table in filesystem format!
emmelaich 3y ago
Does this mean we'll get a good ext2{3,4} driver for Windows?
- _ZeD_ 3y ago
  yeah, I was wondering it too
- Dwedit 3y ago
  Ext2ifs is ancient software.
  - mminer237 3y ago
    It can't handle modern ext4 though. Last time I tried it, it completely corrupted my drive.
- NavinF 3y ago
  WSL2 can do that
  - jesuslop 3y ago
    At leat latests versions, coming from the store or just baked from https://github.com/microsoft/WSL/releases https://github.com/microsoft/WSL/releases. As per [1] One invokes GET-CimInstance -query "SELECT * from Win32_DiskDrive" in Powershell to list devices and then, wsl --mount "\\.\PHYSICALDRIVE1" --bare or whatever. [1] https://learn.microsoft.com/en-us/windows/wsl/wsl2-mount-disk https://learn.microsoft.com/en-us/windows/wsl/wsl2-mount-dis...
veave 3y ago
I use this along with https://github.com/winfsp/sshfs-win https://github.com/winfsp/sshfs-win, which allows me to mount ssh filesystems as windows network shares, and it's awfully slow. So slow that when doing operations that involve opening several files (like using git) or even listing files (like Ctrl+P in vscode) you want to punch your screen. Anybody knows of an alternative?
- angry_octet 3y ago
  This is not my experience. Are you connecting over a LAN or a lossy WAN? Do you have EDR/AV that is hooking every system call?
nickcw 3y ago
Rclone uses WinFSP on windows to mount cloud storage systems via the FUSE compatibility layer. This enables one implementation of a FUSE filing system to work on lots of OSes. Bill Zissimopoulos also wrote cgofuse which rclone uses to access the FUSE compatibility layer. It's a FUSE library for Go. We use that on Windows and macOS (via macFUSE or fuse-T) too. It runs on Linux too but isn't the default as it needs CGO (linking with C). I've worked a lot with Bill and he knows more about Windows file systems and Windows internals than anyone I've ever met!
- NicoJuicy 3y ago
  Yeah, i use rclone to mount my box account to my z:/ drive. Really handy for my backups that are stored there, it's a bit slow though. But that's why there is a cache.
  - BoppreH 3y ago
    Do you have any protections against ransomware overwiting your backups too? That's the only thing keeping me from using a solution like this.
    - mmwelt 3y ago
      I'm not the person you asked, but one way would be to mount the backup drive just for the backup, then unmount it once the backup is complete. This would limit the time the backup is exposed to ransomware.
    - rainbowzootsuit 3y ago
      I'm unfamiliar with Box, but many of these services have versioning/snapshot on the server side and would allow rolling back changes.
    - NicoJuicy 3y ago
      "Yes". I backup to the day folder ,( eg. 01-31) And to the latest folder
    - 3y ago
HeckFeck 3y ago
Much as I am a *nix man through and through, I'm always impressed to see how Windows can be hacked through clever exploitation of its modularity.
EdSchouten 3y ago
I know WinFsp has been around for quite some time, but I wonder how it compares against Microsoft's own ProjFS: https://learn.microsoft.com/en-us/windows/win32/projfs/projected-file-system https://learn.microsoft.com/en-us/windows/win32/projfs/proje...
- larschdk 3y ago
  ProjFS can solve some of the same use cases, but is different by design. It is basically local file storage with a virtual backend that can provide files and data to the local file system. Once files are local, the provider is not asked when files are accessed or modified. The provider can monitor changes and synchronize them with the backend (with chance of conflicts). The provider also needs to manually update the local view when the backing store has changes. It appears to be built for OneDrive. It is way more efficient for many use cases, but also requires more effort to synchronize and resolve conflicts.
oaiey 3y ago
The samples always show mounting similar to a (network) drive. What about mounting in file explorer like OneDrive?
- oaiey 3y ago
  Seems there is a Windows native API for cloud syncs. https://learn.microsoft.com/en-us/windows/win32/cfapi/build-a-cloud-file-sync-engine https://learn.microsoft.com/en-us/windows/win32/cfapi/build-...
SillyUsername 3y ago
Apache Cassandra + this would give high availability and reliability by being able to add/remove nodes randomly. Think transparent backups like RAID 1 or depending on how you structure the data you could even use joins to do RAID 0 and get a performance boost. Obviously this won't be as fast a real RAID cluster but throwing random storage at it like old usb sticks etc might be a bit of fun and an easy way to expand it over time to bigger devices. select * from files where filePath="d:\data"
- lostmsu 3y ago
  You can already do that with Storage Spaces.

151 points by mariuz 3y ago

I wonder how this compares to Dokan. Dokan doesn't integrate into the Windows Cache Manager, and file performance is constrained vs reading something which was recently read.

jarym 3y ago
I’d like to know this too - Dokan wasn’t the most stable when I used it years ago (maybe it’s improved since then), would love to know how this is different
- Dwedit 3y ago
  I tried out Dokan a couple years ago, and it seemed decently stable, just underperforming. There was a big fix that tremendously increased performance and also fixed a race condition around that time. On a completely different note, there's also "Windows Projected File System", which sounds a lot like a user-mode file system, but it's not. It requires that files be backed by real files on your disk.
  - themerone 3y ago
    By my reading of the docs, Windows Projected File System (ProjFS) only requires reliability and performance characteristics comparable to a local filesystem. I think it would work if your data is in memory or backed by a 10G+ lan.
  - accoil 3y ago
    Yeah, that was a surprise to me when I toyed with Windows Projected File System. Turned off the program, and all my "pretend" files were still there.

ehPReth 3y ago

rclone uses this! it’s great!

noisem4ker 3y ago
Another neat application is SSHFS-Win, from the same authors of WinFsp itself. https://github.com/winfsp/sshfs-win https://github.com/winfsp/sshfs-win

MuffinFlavored 3y ago

This is super cool. Why would somebody need to use this instead of like, smb or however normal Windows machines on the same network share drives/files/folders (even with other Mac machines on the network, or Linux, etc.)?

Dwedit 3y ago
This isn't for network shares, this is for reading/writing programmatically generated files. The possibilities are endless. For example, you could make a "filesystem" where everything inside looks like an uncompressed BMP file, but the actual files backing them are some other image format. Then ancient software would be able to use the newest image formats.
themerone 3y ago
That is a reasonable approach, one of the 2 MacOS fuse implementations uses NFS. I've never written an SMB server, but I assume it is a lot more complicated than using WinFsp or Fuse. WSL 2 uses the 9P network filesystem to share files between Linux & Windows. Supposedly this is a simple protocol to implement. I've searched, but can't find any documentation on using 9P without WSL.
- hrunt 3y ago
  Cryptomator suggests WinFSP for its Windows installation, MacFUSE for Intel Macs, and FUSE-T (the NFS FUSE implementation) for Apple silicon Macs. I've used all of them, and I've also used Dokan with encfs. Both OSes treat network filesystems differently from local disks. The FUSE-T implementation is very speedy, but it shows up in the finder as a connection to "localhost", not as a properly named mount. In addition, macOS creates .DS_Store files on network drives by default (there is a setting you can change from the CLI to prevent this). On Windows, the filesystems can be mounted as local drives or network drives. When mounted as network drives, they can have issues with indexing and trust. All-in-all, drives you expect to function as local drives work best when they're actually local device filesystems and not network mounts.
ww520 3y ago
This is for exposing other data as a virtual file system. Let's say you want to expose the Git versions as folders and files. You need this to write the file system layer on top of Git.

nashashmi 3y ago

Database Table in filesystem format!

emmelaich 3y ago

Does this mean we'll get a good ext2{3,4} driver for Windows?

_ZeD_ 3y ago
yeah, I was wondering it too
Dwedit 3y ago
Ext2ifs is ancient software.
- mminer237 3y ago
  It can't handle modern ext4 though. Last time I tried it, it completely corrupted my drive.
NavinF 3y ago
WSL2 can do that
- jesuslop 3y ago
  At leat latests versions, coming from the store or just baked from https://github.com/microsoft/WSL/releases https://github.com/microsoft/WSL/releases. As per [1] One invokes GET-CimInstance -query "SELECT * from Win32_DiskDrive" in Powershell to list devices and then, wsl --mount "\\.\PHYSICALDRIVE1" --bare or whatever. [1] https://learn.microsoft.com/en-us/windows/wsl/wsl2-mount-disk https://learn.microsoft.com/en-us/windows/wsl/wsl2-mount-dis...

veave 3y ago

I use this along with https://github.com/winfsp/sshfs-win https://github.com/winfsp/sshfs-win, which allows me to mount ssh filesystems as windows network shares, and it's awfully slow. So slow that when doing operations that involve opening several files (like using git) or even listing files (like Ctrl+P in vscode) you want to punch your screen. Anybody knows of an alternative?

angry_octet 3y ago
This is not my experience. Are you connecting over a LAN or a lossy WAN? Do you have EDR/AV that is hooking every system call?

nickcw 3y ago

Rclone uses WinFSP on windows to mount cloud storage systems via the FUSE compatibility layer. This enables one implementation of a FUSE filing system to work on lots of OSes. Bill Zissimopoulos also wrote cgofuse which rclone uses to access the FUSE compatibility layer. It's a FUSE library for Go. We use that on Windows and macOS (via macFUSE or fuse-T) too. It runs on Linux too but isn't the default as it needs CGO (linking with C). I've worked a lot with Bill and he knows more about Windows file systems and Windows internals than anyone I've ever met!

NicoJuicy 3y ago
Yeah, i use rclone to mount my box account to my z:/ drive. Really handy for my backups that are stored there, it's a bit slow though. But that's why there is a cache.
- BoppreH 3y ago
  Do you have any protections against ransomware overwiting your backups too? That's the only thing keeping me from using a solution like this.
  - mmwelt 3y ago
    I'm not the person you asked, but one way would be to mount the backup drive just for the backup, then unmount it once the backup is complete. This would limit the time the backup is exposed to ransomware.
  - rainbowzootsuit 3y ago
    I'm unfamiliar with Box, but many of these services have versioning/snapshot on the server side and would allow rolling back changes.
  - NicoJuicy 3y ago
    "Yes". I backup to the day folder ,( eg. 01-31) And to the latest folder
  - 3y ago

HeckFeck 3y ago

Much as I am a *nix man through and through, I'm always impressed to see how Windows can be hacked through clever exploitation of its modularity.

EdSchouten 3y ago

I know WinFsp has been around for quite some time, but I wonder how it compares against Microsoft's own ProjFS: https://learn.microsoft.com/en-us/windows/win32/projfs/projected-file-system https://learn.microsoft.com/en-us/windows/win32/projfs/proje...

larschdk 3y ago
ProjFS can solve some of the same use cases, but is different by design. It is basically local file storage with a virtual backend that can provide files and data to the local file system. Once files are local, the provider is not asked when files are accessed or modified. The provider can monitor changes and synchronize them with the backend (with chance of conflicts). The provider also needs to manually update the local view when the backing store has changes. It appears to be built for OneDrive. It is way more efficient for many use cases, but also requires more effort to synchronize and resolve conflicts.

oaiey 3y ago

The samples always show mounting similar to a (network) drive. What about mounting in file explorer like OneDrive?

oaiey 3y ago
Seems there is a Windows native API for cloud syncs. https://learn.microsoft.com/en-us/windows/win32/cfapi/build-a-cloud-file-sync-engine https://learn.microsoft.com/en-us/windows/win32/cfapi/build-...

SillyUsername 3y ago

Apache Cassandra + this would give high availability and reliability by being able to add/remove nodes randomly. Think transparent backups like RAID 1 or depending on how you structure the data you could even use joins to do RAID 0 and get a performance boost. Obviously this won't be as fast a real RAID cluster but throwing random storage at it like old usb sticks etc might be a bit of fun and an easy way to expand it over time to bigger devices. select * from files where filePath="d:\data"

lostmsu 3y ago
You can already do that with Storage Spaces.