8 ms·
Webfs: A Filesystem Built on Top of the Web
- tootie 7y agoRemember WebDAV? It was a similar concept, but never really found its footing and most of the implementations were pretty shaky. I always thought it was a good idea though. https://en.wikipedia.org/wiki/WebDAV https://en.wikipedia.org/wiki/WebDAV
- Eikon 7y agoThe rfc is super dumb though. For instance, when handling a PROPFIND request (more or less listing files / folders), it’s not mandatory for the server to honor the Depth header (how many levels are returned). There is also no mechanism for the server to advertise whether or not it’s honoring the Depth header. That means the Depth header is useless because the client has no way to know whether there was only one hierarchy depth or if the server did not honor the Depth header. Therefore, your only option is to always scan the full hierarchy using PROPFIND at each level. The RFC is full of that kind of crazy gotchas, not to mention the overuse of “MAY” or “SHOULD” which will drive you crazy if you try to implement a client / server. If you want to go further in insanity, just look at how crazily over engineered the locking mechanism is. I have no words for it. https://tools.ietf.org/html/rfc4918#section-6 https://tools.ietf.org/html/rfc4918#section-6 Unfortunately, even if you think you implemented the whole rfc correctly, your implementation will work with almost nothing as not that much implementations are any good in the wild. A useful WebDAV implementation must be full of vendor-specific workarounds.
- tootie 7y agoThere was an even worse protocol back then called CMIS. It was an attempt to define a standard API for content management which turned into this absolute enterprise monstrosity that would make SOAP blush. It was also impossible to implement. It's interesting looking back but I think developing a standard has a higher chance of success coming from some dude's GitHub than it does with $1T of market cap behind it.
- deleted 7y ago[deleted]
- jpalomaki 7y agoLimitations on depth makes sense, because the actual storage implementation can make recursive retrievals very costly. A folder could be an abstraction for a remote resource. Of course it would still make sense for server to tell client about this (there are files/no files/I don’t know).
- paulddraper 7y agoAgreed. The spec is overcomplicated and half broken.
- amelius 7y agoI always thought WebDAV was sabotaged by different players for some reason (e.g. Apple's implementation caused loss of data). "Never attribute to malice", but support was so bad that almost no other conclusion was possible.
- cryptoz 7y ago"Never attribute to malice what can be adequately explained by stupidity" has led me astray so many times in life I've come to largely disbelieve it is a useful mantra. It lets bad actors hide behind stupidity and cause chaos on purpose, while good people let it happen because "it's simply accidental, right"?
- sitkack 7y agoI think it is slightly more nuanced, in that bad decisions get made on accident and overlooked on purpose. If you want to sabotage something, put lots of people that make many accidental mistakes on a project and put your people in a position to overlook them.
- jlokier 7y agoMaybe it's just Apple competence in this area. Apple's SMB client causes loss of data as well.
- CharlesW 7y agoI've never experienced that (and it's the only way I access my NAS), but you have my attention. Citation(s) please?
- mlyle 7y agoEvery single major operating system ships with WebDAV remote filesystem support built in and it works reasonably. Subversion is built out of WebDAV and can be wired to naively auto-commit changes these clients store. Maybe it's on the decline, but I'd hardly put it as something that "never really found its footing". It is still a decent way to do fileshares over the public internet without sshfs, etc. That's not really what the link is, though. It's an adaptation layer to turn a random network resource into something that looks like a filesystem.
- brendoncarroll 7y agoA WebDAV file could be used as a WebFS Cell. The locking would allow emulation of the compare-and-swap functionality. The reference HTTP Cell: client and server are way less complicated than WebDAV though, as many here have alluded to.
- zzo38computer 7y agoSomeone else described my "httpdirlist" specification as "like WebDAV but better" or "like WebDAV but less messy" (actually I do not remember the exact wording). Now I see that Wikipedia also lists several alternatives to WebDAV too, but I think httpdirlist is good.
- latchkey 7y agoAh yes... I implemented Sardine [1] 10 years ago! At the time, I worked for a large porn company and we couldn't host stuff 'in the cloud' because they didn't allow porn there. We invested a ton of money into an Isilon NAS to store our image/video content and the best way to get stuff off it over HTTP was via webdav. Unfortunately, there wasn't a good Java client. So, I built a simple proxy that would accept regular GET requests and on the back end, use webdav to retrieve content from the Isilon. In front of that proxy was our CDN. Since then, Sardine has been the basis for quite a few other projects. [1] https://github.com/lookfirst/sardine https://github.com/lookfirst/sardine
- thomasfl 7y agoI implemented tve same for ruby. With monkey patching you write normal code for writing files, but use the WebDAV protcol to write files if files had names starting with http.
- Nux 7y agoI use it every day on Android, Windows and Linux. It found its footing.
- brendoncarroll 7y agoHey everyone, I'm the author of WebFS. Happy to answer any questions.
- jerry292 7y agoI read you use IPFS. To my knowledge everything on there can be encrypted however not private. Is there a specific way you get around that.
- brendoncarroll 7y agoYes. WebFS doesn't actually use any of the file/directory functionality provided by IPFS, or any encryption features. We only use the get/put block functionality. Everything is encrypted in WebFS before being posted to a Store. The data encryption keys are generated using a secret and the hash of the data being encrypted. That key is stored in the reference to that data. This continues recursively to the superblock which is not encrypted.
- williesleg 7y agoI want to use Twitter and Facebook feeds to house my data. That would be fantastic!
- austincheney 7y agoI am working on a similar idea myself. Here is a video demo from a month ago: http://mailmarkup.org/sharefile/demo1.mp4 http://mailmarkup.org/sharefile/demo1.mp4
- dredmorbius 7y agoWatching now. 1. Thanks for the direct mp4 link. 2. Consider reducing your desktop resolution for webcasts. I can see that there are dialogues open. I can't for the life of me see what's presented in them.
- cryptonector 7y agoHTTP is basically a filesystem protocol that supports magical files -- not too unlike sharing named pipes over SMB. Not too unlike what it'd be like if one could open(2) AF_LOCAL sockets on Unix/POSIX systems instead of having to connect(2) to them -- if that had been so in 1982 in BSD, it would be true now, NFS would have supported the same, etc.