8 ms·

Windows File System Proxy – FUSE-Like Capability for Windows

89 points by voltagex_ 10y ago

skrowl 10y ago
Is it like https://dokan-dev.github.io/ https://dokan-dev.github.io/ ? If not, how does it differ?
- malkia 10y ago
  There is also CBFS from Eldos -> https://www.eldos.com/cbfs/ https://www.eldos.com/cbfs/ - (commercial)
- malkia 10y ago
  For comparison probably a good start is the API (I'm somewhat familiar with dokan, used it several years ago for some project) https://github.com/billziss-gh/winfsp/blob/master/inc/winfsp/winfsp.h https://github.com/billziss-gh/winfsp/blob/master/inc/winfsp... https://github.com/dokan-dev/dokany/blob/master/dokan/dokan.h https://github.com/dokan-dev/dokany/blob/master/dokan/dokan.... It's really the edge cases that one might run into, where things no longer work well, but I don't recall right now into what problems I've run into (What I did was for the xbox360 devkit to read files from my HDD, rather to copy to it, as was required by the sdk, but I've achieved only 5mb/s, while copying was 10mb/s - the only benefit was if you wanted to read on demand, a bit and for fun). Sony later used dokan for their devkit in PS4 I think? just found some info here: http://www.psdevwiki.com/ps4/File_System_Driver http://www.psdevwiki.com/ps4/File_System_Driver
- Rondom 10y ago
  I have not looked at the technical side, but one big thing is the license. Dokan is LGPL (the library is LGPL, the driver GPL). This one is released under the AGPL, though. :-(
int_19h 10y ago
I'm curious whether anyone had ever tried implementing a pure userspace solution (without a kernel driver) by basically exposing the underlying FS as a CIFS share on some random local port. Obviously, that would be slower, but for many applications that doesn't matter much - while the ability to install and run without administrator privileges would be valuable.
- lukeh 10y ago
  OpenAFS used to do this if I remember right. They moved to an IFS.
- saynsedit 10y ago
  I've done this, it works.
  - voltagex_ 10y ago
    On Windows? I didn't think Windows would connect to SMB/CIFS on any other ports other than 139/445. Can you share any other details?
    - tiraniddo 10y ago
      Not sure you can do it easily with SMB/CIFS (maybe some NAT tricks) but depending on the level of FS support you want it's pretty easy to do it with WebDav. You can mount a webdav share pointing to localhost on an arbitrary port (I believe it's of the work \\localhost@portno\DAVWWWRoot). From there I'm sure you could proxy out to any other FS implementation you liked. It also probably isn't great for large files etc due to the way WebDAV works of course :-)
    - saynsedit 10y ago
      For this particular client we had Administrator access. SMB was used since we did not trust Dokan and did not want to pay for CBFS.
- ComodoHacker 10y ago
  How do I connect to CIFS on non-default port?
  - captainmuon 10y ago
    The only way I found was to use (cygwin ssh) port forwarding to map that port to the standard one, kill the default network SMB process (which restarts), and hope my process wins the race. Then you could access the remote shares at \\127.0.0.1 . I think I also managed to create a loopback device, put it into its own subnet, and map the port to that device, and then it was possible to do e.g. \\192.168.100.56, but it was a pain to set up. The reason I needed this was, I wanted to run samba as non-root on a remote host (on a non-privileged port), and forwarded that with ssh though a firewall. This way I could access my files on the remote Linux server from my windows desktop. One problem is that windows does not tolerate hanging connections much, and a bad or lost connecting can freeze your program or require a reboot. I'm not sure how much is the kernels fault and how much you can code around this.
jessedhillon 10y ago
I'm wondering what the rationale is for the usage of `goto` in the C source? It looks like it's exclusively used to implement a pattern of exiting or failing early and in a common way. Is this something that's common in modern C?
- saynsedit 10y ago
  Yes, goto exit and cleanup resources on error is a common and not frowned upon idiom in C. See Python source.
  - creshal 10y ago
    Or the Linux kernel. (Or Apple's "goto fail" bug for a somewhat infamous example how to not use it.)
    - krylon 10y ago
      > Or Apple's "goto fail" bug for a somewhat infamous example how to not use it. But that problem was not because of the goto, strictly speaking.
- rossy 10y ago
  I'm not entirely sure how common it is in modern projects, but I use it a lot. I find functions that use this kind of centralized cleanup code a lot easier to read than functions that avoid 'goto' by duplicating the cleanup code or handling errors with deep nested if blocks. The Linux kernel coding style guide recommends it as well (see "Centralized exiting of functions" in https://www.kernel.org/doc/Documentation/CodingStyle https://www.kernel.org/doc/Documentation/CodingStyle) FWIW, it seems to be the primary use of 'goto' in these projects: https://github.com/torvalds/linux/search?q=goto https://github.com/torvalds/linux/search?q=goto https://github.com/FFmpeg/FFmpeg/search?q=goto https://github.com/FFmpeg/FFmpeg/search?q=goto https://github.com/antirez/redis/search?q=goto https://github.com/antirez/redis/search?q=goto
- mhogomchungu 10y ago
  I dont use "goto" in C but a specialized static function that does more or less the same thing. Example is the zuluExit "goto" function: https://github.com/mhogomchungu/zuluCrypt/blob/master/zuluCrypt-cli/lib/open_volume.c https://github.com/mhogomchungu/zuluCrypt/blob/master/zuluCr...
jorangreef 10y ago
"WinFsp is available under the AGPLv3 license. If you find the constraints of the AGPLv3 too onerous, a commercial license is also available. Please contact Bill Zissimopoulos for more details." I hope they will soon switch to MIT.
- m_mueller 10y ago
  "Hey, could you make your free thing even free-er so I can make money with it in your place?"
  - drdaeman 10y ago
    "...so I could make a non-free derivative/build?" Fixed it for you. A lot of people make money by working (or integrating) strongly copylefted software, with results remaining friendly to end-user freedoms.
    - m_mueller 10y ago
      I'm curious. What are examples that are not based around offering what the software does as a service (including support / consulting based products)?
      - drdaeman 10y ago
        I believe, every day, a lot of FOSS projects get forked by engineers (who are getting paid for their work). They add new features or fix bugs they've encountered, and share their work results back upstream. Or release as their own fork/patchset (I stumble upon such forks almost every day I browse GitHub/Bitbucket/GitLab). Say... for an example that resembles the OP project... In a company I work for, we have a FUSE-based (GNU/Linux) filesystem that we use to access Azure Blob Storage like any other mounted network share. A puny tiny project that's a fork of someone else's fork of someone's project, all FOSS (Apache2-licensed, though, not *GPL, but I believe it's irrelevant here), but hope it should still suffice as a quick example. There's literally no reason for us to keep it secret - we're end-user ourselves, enjoying cp and rsync working nicely. Well, maybe not completely nicely - I'm aware there are still some issues with that filesystem, but at least, with the patches, it works OK for us. So, sure, we've put all our changes on GitHub. And, I'm sure, every our developer who had touched the code to solve the problems we had with it got paid for their time. :) So, I guess, it's every time the engineer is an employee of a company that's an end-user themselves. I think the only case where GPL/AGPL doesn't fit is when you license the software to the end-users to run, and you don't want them to re-distribute and modify software on their own but rely on you for all their needs. And it seems that original project author doesn't fancy this particular business model.
ZoomZoomZoom 10y ago
For those interested in the same workflow demonstrated on demo GIF, there's Win-SSHfs, built on Dokany. This fork works pretty well: https://github.com/mfrnd/win-sshfs https://github.com/mfrnd/win-sshfs
ericfrederich 10y ago
Why not FUSE itself? I wonder if the Windows Subsystem for Linux guys could get FUSE working.
- billziss 10y ago
  WinFsp does have a FUSE API. It implements the FUSE 2.8 (high-level) API. This is in addition to its native API. I also have a FUSE package submission to Cygwin. It needs votes :) https://cygwin.com/ml/cygwin-apps/2016-07/msg00041.html https://cygwin.com/ml/cygwin-apps/2016-07/msg00041.html Bill Zissimopoulos
billziss 10y ago
I am late to the party as I just found out about this discussion. I am the author of WinFsp. You can ask me specific questions if you want. Bill Zissimopoulos

89 points by voltagex_ 10y ago

skrowl 10y ago

Is it like https://dokan-dev.github.io/ https://dokan-dev.github.io/ ? If not, how does it differ?

malkia 10y ago
There is also CBFS from Eldos -> https://www.eldos.com/cbfs/ https://www.eldos.com/cbfs/ - (commercial)
malkia 10y ago
For comparison probably a good start is the API (I'm somewhat familiar with dokan, used it several years ago for some project) https://github.com/billziss-gh/winfsp/blob/master/inc/winfsp/winfsp.h https://github.com/billziss-gh/winfsp/blob/master/inc/winfsp... https://github.com/dokan-dev/dokany/blob/master/dokan/dokan.h https://github.com/dokan-dev/dokany/blob/master/dokan/dokan.... It's really the edge cases that one might run into, where things no longer work well, but I don't recall right now into what problems I've run into (What I did was for the xbox360 devkit to read files from my HDD, rather to copy to it, as was required by the sdk, but I've achieved only 5mb/s, while copying was 10mb/s - the only benefit was if you wanted to read on demand, a bit and for fun). Sony later used dokan for their devkit in PS4 I think? just found some info here: http://www.psdevwiki.com/ps4/File_System_Driver http://www.psdevwiki.com/ps4/File_System_Driver
Rondom 10y ago
I have not looked at the technical side, but one big thing is the license. Dokan is LGPL (the library is LGPL, the driver GPL). This one is released under the AGPL, though. :-(

int_19h 10y ago

I'm curious whether anyone had ever tried implementing a pure userspace solution (without a kernel driver) by basically exposing the underlying FS as a CIFS share on some random local port. Obviously, that would be slower, but for many applications that doesn't matter much - while the ability to install and run without administrator privileges would be valuable.

lukeh 10y ago
OpenAFS used to do this if I remember right. They moved to an IFS.
saynsedit 10y ago
I've done this, it works.
- voltagex_ 10y ago
  On Windows? I didn't think Windows would connect to SMB/CIFS on any other ports other than 139/445. Can you share any other details?
  - tiraniddo 10y ago
    Not sure you can do it easily with SMB/CIFS (maybe some NAT tricks) but depending on the level of FS support you want it's pretty easy to do it with WebDav. You can mount a webdav share pointing to localhost on an arbitrary port (I believe it's of the work \\localhost@portno\DAVWWWRoot). From there I'm sure you could proxy out to any other FS implementation you liked. It also probably isn't great for large files etc due to the way WebDAV works of course :-)
  - saynsedit 10y ago
    For this particular client we had Administrator access. SMB was used since we did not trust Dokan and did not want to pay for CBFS.
ComodoHacker 10y ago
How do I connect to CIFS on non-default port?
- captainmuon 10y ago
  The only way I found was to use (cygwin ssh) port forwarding to map that port to the standard one, kill the default network SMB process (which restarts), and hope my process wins the race. Then you could access the remote shares at \\127.0.0.1 . I think I also managed to create a loopback device, put it into its own subnet, and map the port to that device, and then it was possible to do e.g. \\192.168.100.56, but it was a pain to set up. The reason I needed this was, I wanted to run samba as non-root on a remote host (on a non-privileged port), and forwarded that with ssh though a firewall. This way I could access my files on the remote Linux server from my windows desktop. One problem is that windows does not tolerate hanging connections much, and a bad or lost connecting can freeze your program or require a reboot. I'm not sure how much is the kernels fault and how much you can code around this.

jessedhillon 10y ago

I'm wondering what the rationale is for the usage of `goto` in the C source? It looks like it's exclusively used to implement a pattern of exiting or failing early and in a common way. Is this something that's common in modern C?

saynsedit 10y ago
Yes, goto exit and cleanup resources on error is a common and not frowned upon idiom in C. See Python source.
- creshal 10y ago
  Or the Linux kernel. (Or Apple's "goto fail" bug for a somewhat infamous example how to not use it.)
  - krylon 10y ago
    > Or Apple's "goto fail" bug for a somewhat infamous example how to not use it. But that problem was not because of the goto, strictly speaking.
rossy 10y ago
I'm not entirely sure how common it is in modern projects, but I use it a lot. I find functions that use this kind of centralized cleanup code a lot easier to read than functions that avoid 'goto' by duplicating the cleanup code or handling errors with deep nested if blocks. The Linux kernel coding style guide recommends it as well (see "Centralized exiting of functions" in https://www.kernel.org/doc/Documentation/CodingStyle https://www.kernel.org/doc/Documentation/CodingStyle) FWIW, it seems to be the primary use of 'goto' in these projects: https://github.com/torvalds/linux/search?q=goto https://github.com/torvalds/linux/search?q=goto https://github.com/FFmpeg/FFmpeg/search?q=goto https://github.com/FFmpeg/FFmpeg/search?q=goto https://github.com/antirez/redis/search?q=goto https://github.com/antirez/redis/search?q=goto
mhogomchungu 10y ago
I dont use "goto" in C but a specialized static function that does more or less the same thing. Example is the zuluExit "goto" function: https://github.com/mhogomchungu/zuluCrypt/blob/master/zuluCrypt-cli/lib/open_volume.c https://github.com/mhogomchungu/zuluCrypt/blob/master/zuluCr...

jorangreef 10y ago

"WinFsp is available under the AGPLv3 license. If you find the constraints of the AGPLv3 too onerous, a commercial license is also available. Please contact Bill Zissimopoulos for more details." I hope they will soon switch to MIT.

m_mueller 10y ago
"Hey, could you make your free thing even free-er so I can make money with it in your place?"
- drdaeman 10y ago
  "...so I could make a non-free derivative/build?" Fixed it for you. A lot of people make money by working (or integrating) strongly copylefted software, with results remaining friendly to end-user freedoms.
  - m_mueller 10y ago
    I'm curious. What are examples that are not based around offering what the software does as a service (including support / consulting based products)?
    - drdaeman 10y ago
      I believe, every day, a lot of FOSS projects get forked by engineers (who are getting paid for their work). They add new features or fix bugs they've encountered, and share their work results back upstream. Or release as their own fork/patchset (I stumble upon such forks almost every day I browse GitHub/Bitbucket/GitLab). Say... for an example that resembles the OP project... In a company I work for, we have a FUSE-based (GNU/Linux) filesystem that we use to access Azure Blob Storage like any other mounted network share. A puny tiny project that's a fork of someone else's fork of someone's project, all FOSS (Apache2-licensed, though, not *GPL, but I believe it's irrelevant here), but hope it should still suffice as a quick example. There's literally no reason for us to keep it secret - we're end-user ourselves, enjoying cp and rsync working nicely. Well, maybe not completely nicely - I'm aware there are still some issues with that filesystem, but at least, with the patches, it works OK for us. So, sure, we've put all our changes on GitHub. And, I'm sure, every our developer who had touched the code to solve the problems we had with it got paid for their time. :) So, I guess, it's every time the engineer is an employee of a company that's an end-user themselves. I think the only case where GPL/AGPL doesn't fit is when you license the software to the end-users to run, and you don't want them to re-distribute and modify software on their own but rely on you for all their needs. And it seems that original project author doesn't fancy this particular business model.

ZoomZoomZoom 10y ago

For those interested in the same workflow demonstrated on demo GIF, there's Win-SSHfs, built on Dokany. This fork works pretty well: https://github.com/mfrnd/win-sshfs https://github.com/mfrnd/win-sshfs

ericfrederich 10y ago

Why not FUSE itself? I wonder if the Windows Subsystem for Linux guys could get FUSE working.

billziss 10y ago
WinFsp does have a FUSE API. It implements the FUSE 2.8 (high-level) API. This is in addition to its native API. I also have a FUSE package submission to Cygwin. It needs votes :) https://cygwin.com/ml/cygwin-apps/2016-07/msg00041.html https://cygwin.com/ml/cygwin-apps/2016-07/msg00041.html Bill Zissimopoulos

billziss 10y ago

I am late to the party as I just found out about this discussion. I am the author of WinFsp. You can ask me specific questions if you want. Bill Zissimopoulos