7 ms·

WinFsp – Windows File System Proxy

184 points by mkotowski 5y ago

hdjjhhvvhga 5y ago
In case anyone is wondering: if you want to use sshfs network drives as demoed on their Github page, WinFsp is not enough, you need to also install SSHFS-Win: https://github.com/billziss-gh/sshfs-win/releases/tag/v3.5.20357 https://github.com/billziss-gh/sshfs-win/releases/tag/v3.5.2...
- juangacovas 5y ago
  Haven't tried the latest versions of WinFsp + SSHFS-Win, glad to see a new GUI for the later and new versions. My experience is good but if you have an unstable connection, the mapped drive didn't reconnect by itself at all times. For a better experience I've always relied on RaiDrive (free with Ads) or SFTP Drive from N software (free for non-commercial)
Wicher 5y ago
Could one easily distribute software that bundles this and have it installed by novice end users? What kind of permission granting seances would they have to go through, if any?
- catwell 5y ago
  You probably can and it won't be nearly as bad as what it is on MacOS. You only need a code signing certificate for drivers from Microsoft. You might not even need that if WinFSP provides signed binaries and you don't modify the driver part. (Source: until 2 years ago I used to work at a company that did that using a proprietary driver - CBFS - and was looking at WinFSP to replace it.)
  - jkaplowitz 5y ago
    > You might not even need that if WinFSP provides signed binaries and you don't modify the driver part. The README.md which the submission linked says that they do indeed provide signed binaries for every release.
- mkotowski 5y ago
  As for using it to create a new file system, the repository has a basic tutorial for that: https://github.com/billziss-gh/winfsp/blob/master/doc/WinFsp-Tutorial.asciidoc https://github.com/billziss-gh/winfsp/blob/master/doc/WinFsp... > What kind of permission granting seances would they have to go through, if any? From personal experience of using this: WinFSP have ready .msi installer. After going through the standard process of installing it, then you need to install the file system implementation you want (for example SSHFS from http://www.secfs.net/winfsp/rel/ http://www.secfs.net/winfsp/rel/). The required permissions amount to the standard installation prompt, at least from my experience on Windows 10.
andix 5y ago
Are there any ready to use file system implementations available?
- icegreentea2 5y ago
  http://www.secfs.net/winfsp/doc/Known-File-Systems/ http://www.secfs.net/winfsp/doc/Known-File-Systems/ I've personally only used SSHFS (years ago). It was ok - slightly flakey for me. I can't remember what was wonky, but it was like 98% working, I couldn't figure out the last bit, so I just switched to straight up SMB/samba (I had pretty free control of the remote end, so that was a reasonable solution). Maybe I'll give SSHFS another try again now...
  - orhmeh09 5y ago
    SSHFS has behaved a little jankily for me on Linux, too, since forever. I wonder if the protocol’s just not really up to the 2%.
    - franga2000 5y ago
      Same in my experience, but SFTP itself isn't exactly great either. Especially when your connection is less than ideal and you're working with large-ish files. SMB tunneled over SSH works better in many cases, but of course that has it's own set of problems and is a huge pain to set up.
- deleted 5y ago
  [deleted]
- michael-ax 5y ago
  i'd love one for keepass and https://www.presslabs.com/docs/code/gitfs/ https://www.presslabs.com/docs/code/gitfs/ :)
  - andix 5y ago
    What should that actually do? Expose passwords as files?
BiteCode_dev 5y ago
Oh, it has a FUSE compat layer too, that's pretty damn cool.
- riedel 5y ago
  You can even implement a fuse server in cygwin, so many fuse implementations will straight forward compile from my experience.
oaiey 5y ago
And then it is GPL with exception for libre software or commercial license. I hope they have reasonable commercial licensing.
DerekBickerton 5y ago
This might be handy for passing files to a virtual machine if you don't want to use Virtualbox's 'guest additions' software
- jabiko 5y ago
  Why wouldn't you want to use the VirtualBox guest additions? It's GPLv2 licensed and brings additional features that make using the VM easier.
  - joombaga 5y ago
    You might not want to be locked in. I've used VirtualBox to build systems meant to eventually run elsewhere.
  - franga2000 5y ago
    It's an absolute disaster for one. I haven't actually managed to get shared folders to work reliably in years. It ended up being faster and more reliable to run Samba on the host and connect over a virtual network. Some of the other features of course can't easily be duplicated with other tools and they work decent-ish anyways so there's no reason to try, but shared folders aren't one of them.
    - selfhoster11 5y ago
      They are always rock solid for me on Windows guests. Linux guests are a different matter altogether, but mainly due to permissions.
- riedel 5y ago
  Actually WSL2 might be a better target as it would enable some tweaking of the mapping. Did anyone have success implementing virtio communication through hyperv and mount e.g. a custom p9fs. E.g. to support ownership and attributes. What would be the best p9fs server to start with? I had limited success with p9fs on winfsp from cygwin some time ago due to the problems of mapping effective access rights correctly.
chajath 5y ago
It's interesting to note that WSL uses 9p to mount the filesystem on Windows. Had MSFT taken a more FOSS-friendly approach, we could've had a burgeoning FUSE ecosystem built around 9p.
- WorldMaker 5y ago
  The WSL team in their GitHub have teased experiments with allowing the 9p server to do more outside of WSL and be an open protocol for Windows moving forward. Given Windows' backwards compatibility mandates, you may be able to understand why they are taking a slow and deliberate exploration process to figure out how much that is something they want to support in Windows moving forward (especially if it escapes the realm of "developer tools" where they feel they have more freedom).
solarkraft 5y ago
Cool, the third version of FUSE for Windows …
sam0x17 5y ago
Used this extensively when I was working on BitFort (abandoned project). It's really solid. If you're open source you can use the whole thing no problem. If you're closed source, a commercial license costs around $5k.
api 5y ago
APFS (Apple Filesystem) on MacOS has some kind of file provider API too. Here's one app that uses it: https://www.expandrive.com/strongsync/ https://www.expandrive.com/strongsync/
pjmlp 5y ago
Windows already has similar native APIs, why bother with this?
- AkshitGarg 5y ago
  It also has a FUSE compat layer which maybe useful when porting existing *nix software
  - pjmlp 5y ago
    Ah ok.
- shadowfox 5y ago
  Out of curiosity: what sort of apis are we talking about?
  - jodrellblank 5y ago
    Maybe this one? > "The Windows Projected File System (ProjFS) allows a user-mode application called a "provider" to project hierarchical data from a backing data store into the file system, making it appear as files and directories in the file system. For example, a simple provider could project the Windows registry into the file system, making registry keys and values appear as files and directories, respectively. An example of a more complex provider is VFS for Git, which is used to virtualize very large git repos." - https://docs.microsoft.com/en-us/windows/win32/projfs/projected-file-system https://docs.microsoft.com/en-us/windows/win32/projfs/projec... or this one? > "Starting in Windows 10, version 1709, Windows provides the cloud files API. This API consists of several native Win32 and WinRT APIs that formalize support for cloud sync engines, and handles tasks such as creating and managing placeholder files and directories." - https://docs.microsoft.com/en-us/windows/win32/cfapi/cloud-files-api-portal https://docs.microsoft.com/en-us/windows/win32/cfapi/cloud-f...
    - marwis 5y ago
      Neither is a generic user space vfs API. They are only tailored to some narrow use cases with bunch of limitations. Last time I checked projfs can't stream files from memory while cloudfs can't stream directories from memory (i.e. you have to pre-create placeholder files in local fs). And forget about advanced stuff like mmap
vlovich123 5y ago
In most of the benchmarks down it outperforms NTFS. What’s going on there?
- sam0x17 5y ago
  It turns out it's actually very easy to outperform off-the-shelf file systems depending on what your metric is. I discovered this when I was developing BitFort (abandoned project) which made use of WinFsp on Windows and pure FUSE on Linux. In my case the secret was having all the paths available in a fast in-memory data structure. I was dealing with a remote cloud file system, yet I was able to open up folders and browse around the file system faster than NTFS can with LOCAL file systems because every path and the MIME type is already in memory. For some applications (i.e. if you have trillions of small files) this might be a memory hog but for most I imagine it would be just fine. Seems like something OSes should have a toggleable option for...
  - mikevm 5y ago
    BitFort sounds interesting - is the source code available? Also, what did you use to build your awesome looking resume page? https://resume.sam0x17.dev/ https://resume.sam0x17.dev/ :)
    - notRobot 5y ago
      That page does look pretty awesome!
    - sam0x17 5y ago
      Thank you I actually used a template that was posted on HN years ago https://github.com/Thomashighbaugh/resume https://github.com/Thomashighbaugh/resume
    - sam0x17 5y ago
      I've been toying with the idea of releasing the bitfort source code as open source, but the usefulness is limited as it only worked with a very specific version of Node back in 2018 and the key dependencies (in particular the FUSE one) no longer works. If I were to re-write I would do it in Rust for sure.
_def 5y ago
Is it maybe possible to use ext3/ext4 rw on windows 10 with this?
- Rucadi 5y ago
  You can already do that using WSL2 mount. After mounting an ext3-4 drive, you can RW into it using the file explorer.
  - galkk 5y ago
    Only in insider builds. wsl --mount is not available in current release
coldacid 5y ago
So other than having FUSE compatibility, why this rather than Dokan?
- riedel 5y ago
  https://news.ycombinator.com/item?id=13126987 https://news.ycombinator.com/item?id=13126987
  - coldacid 5y ago
    A benchmark from nearly five years ago means nothing today.
- rkagerer 5y ago
  I played with Dokan in its early days. It was promising, but I found it a little buggy. (Not sure if that remained true once it matured more)
MichaelGroves 5y ago
> It outperforms NTFS in most scenarios (an unfair comparison as NTFS is a disk file system and WinFsp is tested with an in-memory file system) Oh good grief! Let me guess, they first benchmarked it against a ram disk, but the resulting graphs didn't look so impressive so they scrapped that and decided to benchmark it against a disk, which they admit was unfair, just to get the impressive looking graphics.

184 points by mkotowski 5y ago

In case anyone is wondering: if you want to use sshfs network drives as demoed on their Github page, WinFsp is not enough, you need to also install SSHFS-Win: https://github.com/billziss-gh/sshfs-win/releases/tag/v3.5.20357 https://github.com/billziss-gh/sshfs-win/releases/tag/v3.5.2...

juangacovas 5y ago
Haven't tried the latest versions of WinFsp + SSHFS-Win, glad to see a new GUI for the later and new versions. My experience is good but if you have an unstable connection, the mapped drive didn't reconnect by itself at all times. For a better experience I've always relied on RaiDrive (free with Ads) or SFTP Drive from N software (free for non-commercial)

Wicher 5y ago

Could one easily distribute software that bundles this and have it installed by novice end users? What kind of permission granting seances would they have to go through, if any?

catwell 5y ago
You probably can and it won't be nearly as bad as what it is on MacOS. You only need a code signing certificate for drivers from Microsoft. You might not even need that if WinFSP provides signed binaries and you don't modify the driver part. (Source: until 2 years ago I used to work at a company that did that using a proprietary driver - CBFS - and was looking at WinFSP to replace it.)
- jkaplowitz 5y ago
  > You might not even need that if WinFSP provides signed binaries and you don't modify the driver part. The README.md which the submission linked says that they do indeed provide signed binaries for every release.
mkotowski 5y ago
As for using it to create a new file system, the repository has a basic tutorial for that: https://github.com/billziss-gh/winfsp/blob/master/doc/WinFsp-Tutorial.asciidoc https://github.com/billziss-gh/winfsp/blob/master/doc/WinFsp... > What kind of permission granting seances would they have to go through, if any? From personal experience of using this: WinFSP have ready .msi installer. After going through the standard process of installing it, then you need to install the file system implementation you want (for example SSHFS from http://www.secfs.net/winfsp/rel/ http://www.secfs.net/winfsp/rel/). The required permissions amount to the standard installation prompt, at least from my experience on Windows 10.

andix 5y ago

Are there any ready to use file system implementations available?

icegreentea2 5y ago
http://www.secfs.net/winfsp/doc/Known-File-Systems/ http://www.secfs.net/winfsp/doc/Known-File-Systems/ I've personally only used SSHFS (years ago). It was ok - slightly flakey for me. I can't remember what was wonky, but it was like 98% working, I couldn't figure out the last bit, so I just switched to straight up SMB/samba (I had pretty free control of the remote end, so that was a reasonable solution). Maybe I'll give SSHFS another try again now...
- orhmeh09 5y ago
  SSHFS has behaved a little jankily for me on Linux, too, since forever. I wonder if the protocol’s just not really up to the 2%.
  - franga2000 5y ago
    Same in my experience, but SFTP itself isn't exactly great either. Especially when your connection is less than ideal and you're working with large-ish files. SMB tunneled over SSH works better in many cases, but of course that has it's own set of problems and is a huge pain to set up.
deleted 5y ago
[deleted]
michael-ax 5y ago
i'd love one for keepass and https://www.presslabs.com/docs/code/gitfs/ https://www.presslabs.com/docs/code/gitfs/ :)
- andix 5y ago
  What should that actually do? Expose passwords as files?

BiteCode_dev 5y ago

Oh, it has a FUSE compat layer too, that's pretty damn cool.

riedel 5y ago
You can even implement a fuse server in cygwin, so many fuse implementations will straight forward compile from my experience.

oaiey 5y ago

And then it is GPL with exception for libre software or commercial license. I hope they have reasonable commercial licensing.

DerekBickerton 5y ago

This might be handy for passing files to a virtual machine if you don't want to use Virtualbox's 'guest additions' software

jabiko 5y ago
Why wouldn't you want to use the VirtualBox guest additions? It's GPLv2 licensed and brings additional features that make using the VM easier.
- joombaga 5y ago
  You might not want to be locked in. I've used VirtualBox to build systems meant to eventually run elsewhere.
- franga2000 5y ago
  It's an absolute disaster for one. I haven't actually managed to get shared folders to work reliably in years. It ended up being faster and more reliable to run Samba on the host and connect over a virtual network. Some of the other features of course can't easily be duplicated with other tools and they work decent-ish anyways so there's no reason to try, but shared folders aren't one of them.
  - selfhoster11 5y ago
    They are always rock solid for me on Windows guests. Linux guests are a different matter altogether, but mainly due to permissions.
riedel 5y ago
Actually WSL2 might be a better target as it would enable some tweaking of the mapping. Did anyone have success implementing virtio communication through hyperv and mount e.g. a custom p9fs. E.g. to support ownership and attributes. What would be the best p9fs server to start with? I had limited success with p9fs on winfsp from cygwin some time ago due to the problems of mapping effective access rights correctly.

chajath 5y ago

It's interesting to note that WSL uses 9p to mount the filesystem on Windows. Had MSFT taken a more FOSS-friendly approach, we could've had a burgeoning FUSE ecosystem built around 9p.

WorldMaker 5y ago
The WSL team in their GitHub have teased experiments with allowing the 9p server to do more outside of WSL and be an open protocol for Windows moving forward. Given Windows' backwards compatibility mandates, you may be able to understand why they are taking a slow and deliberate exploration process to figure out how much that is something they want to support in Windows moving forward (especially if it escapes the realm of "developer tools" where they feel they have more freedom).

solarkraft 5y ago

Cool, the third version of FUSE for Windows …

sam0x17 5y ago

Used this extensively when I was working on BitFort (abandoned project). It's really solid. If you're open source you can use the whole thing no problem. If you're closed source, a commercial license costs around $5k.

api 5y ago

APFS (Apple Filesystem) on MacOS has some kind of file provider API too. Here's one app that uses it: https://www.expandrive.com/strongsync/ https://www.expandrive.com/strongsync/

pjmlp 5y ago

Windows already has similar native APIs, why bother with this?

AkshitGarg 5y ago
It also has a FUSE compat layer which maybe useful when porting existing *nix software
- pjmlp 5y ago
  Ah ok.
shadowfox 5y ago
Out of curiosity: what sort of apis are we talking about?
- jodrellblank 5y ago
  Maybe this one? > "The Windows Projected File System (ProjFS) allows a user-mode application called a "provider" to project hierarchical data from a backing data store into the file system, making it appear as files and directories in the file system. For example, a simple provider could project the Windows registry into the file system, making registry keys and values appear as files and directories, respectively. An example of a more complex provider is VFS for Git, which is used to virtualize very large git repos." - https://docs.microsoft.com/en-us/windows/win32/projfs/projected-file-system https://docs.microsoft.com/en-us/windows/win32/projfs/projec... or this one? > "Starting in Windows 10, version 1709, Windows provides the cloud files API. This API consists of several native Win32 and WinRT APIs that formalize support for cloud sync engines, and handles tasks such as creating and managing placeholder files and directories." - https://docs.microsoft.com/en-us/windows/win32/cfapi/cloud-files-api-portal https://docs.microsoft.com/en-us/windows/win32/cfapi/cloud-f...
  - marwis 5y ago
    Neither is a generic user space vfs API. They are only tailored to some narrow use cases with bunch of limitations. Last time I checked projfs can't stream files from memory while cloudfs can't stream directories from memory (i.e. you have to pre-create placeholder files in local fs). And forget about advanced stuff like mmap

vlovich123 5y ago

In most of the benchmarks down it outperforms NTFS. What’s going on there?

sam0x17 5y ago
It turns out it's actually very easy to outperform off-the-shelf file systems depending on what your metric is. I discovered this when I was developing BitFort (abandoned project) which made use of WinFsp on Windows and pure FUSE on Linux. In my case the secret was having all the paths available in a fast in-memory data structure. I was dealing with a remote cloud file system, yet I was able to open up folders and browse around the file system faster than NTFS can with LOCAL file systems because every path and the MIME type is already in memory. For some applications (i.e. if you have trillions of small files) this might be a memory hog but for most I imagine it would be just fine. Seems like something OSes should have a toggleable option for...
- mikevm 5y ago
  BitFort sounds interesting - is the source code available? Also, what did you use to build your awesome looking resume page? https://resume.sam0x17.dev/ https://resume.sam0x17.dev/ :)
  - notRobot 5y ago
    That page does look pretty awesome!
  - sam0x17 5y ago
    Thank you I actually used a template that was posted on HN years ago https://github.com/Thomashighbaugh/resume https://github.com/Thomashighbaugh/resume
  - sam0x17 5y ago
    I've been toying with the idea of releasing the bitfort source code as open source, but the usefulness is limited as it only worked with a very specific version of Node back in 2018 and the key dependencies (in particular the FUSE one) no longer works. If I were to re-write I would do it in Rust for sure.

_def 5y ago

Is it maybe possible to use ext3/ext4 rw on windows 10 with this?

Rucadi 5y ago
You can already do that using WSL2 mount. After mounting an ext3-4 drive, you can RW into it using the file explorer.
- galkk 5y ago
  Only in insider builds. wsl --mount is not available in current release

coldacid 5y ago

So other than having FUSE compatibility, why this rather than Dokan?

riedel 5y ago
https://news.ycombinator.com/item?id=13126987 https://news.ycombinator.com/item?id=13126987
- coldacid 5y ago
  A benchmark from nearly five years ago means nothing today.
rkagerer 5y ago
I played with Dokan in its early days. It was promising, but I found it a little buggy. (Not sure if that remained true once it matured more)

MichaelGroves 5y ago

> It outperforms NTFS in most scenarios (an unfair comparison as NTFS is a disk file system and WinFsp is tested with an in-memory file system) Oh good grief! Let me guess, they first benchmarked it against a ram disk, but the resulting graphs didn't look so impressive so they scrapped that and decided to benchmark it against a disk, which they admit was unfair, just to get the impressive looking graphics.