11 ms·
The File Filesystem (2021)
- isoprophlex 2y agoFor fuck's sake! Not everything needs to be a file! (This is a joke. I love the idea and execution.)
- mypalmike 2y agoI see what you did there.
- vidarh 2y agoDaniel Stenberg, of cURL fame, co-write an Amiga-editor called FrexxEd where the open buffers were exposed as files in the filesystem. Meaning you could write any shell script to manipulate an open buffer (not that important as it also exposed all editor functionality both via IPC via Arexx and via FPL - a C-like scripting language), and that you could e.g. compile without saving (that was very helpful on a system where a lot of people might only even have a single floppy drive, and where being able to have the compiler in the drive and compile straight from the in-memory version in RAM so you didn't have to keep swapping floppies was highly useful (just remember to save before actually trying to run the program - no memory protection...)
- salgernon 2y agoClassic MacOS in the 80s had "MPW" macintosh programmers workshop - that treated open text windows as files and selections within the windows as files, so it wasn't uncommon to have a portion of an otherwise documentation file have a "click here and hit enter", which would use the selected text as stdin for some semi-ported unix tool. (no memory protection or multitasking, so true pipelines with backpressure didn't work)
- bombcar 2y agoBBEdit has something akin - you can select text and run a Unix command on the selection via temp files. Very useful
- fsckboy 2y agoI want a webbrowser that does that, lets me shell-cd into each tab as a directory
- freeney 2y agoThis looks awesome, I need to give it a try asap. I can very well see myself using this to navigate or search inside JSON files
- yjftsjthsd-h 2y agoI would gently suggest naming it filefs or something; ffs already means https://man.freebsd.org/cgi/man.cgi?ffs(7) https://man.freebsd.org/cgi/man.cgi?ffs(7) That said - good idea/approach; seems like an excellent way to cleanly extend the unix approach to structured file formats:)
- wegwerff 2y agoalso https://en.wikipedia.org/wiki/Formatted_File_System https://en.wikipedia.org/wiki/Formatted_File_System and https://en.wikipedia.org/wiki/Flash_file_system https://en.wikipedia.org/wiki/Flash_file_system and kind of https://en.wikipedia.org/wiki/Amiga_Fast_File_System https://en.wikipedia.org/wiki/Amiga_Fast_File_System Four file systems named ffs; ffs is popular for file systems
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- galkk 2y agoOp’s ffs do not target FreeBSD and it l seems like referenced system is FreeBSD only. Claiming naming rights is a stretch here
- eichin 2y agoThat's ahistorical. FFS is the berkeley Fast File System, from BSD 4.2, in 1983.
- yjftsjthsd-h 2y agoFFS predates FreeBSD and is in some capacity supported by all 3 major BSDs. I'm fairly confident that Linux actually supports it through the ufs driver ( https://github.com/torvalds/linux/tree/master/fs/ufs https://github.com/torvalds/linux/tree/master/fs/ufs ); whether the use of different names in different places makes it better or worse is an exercise for the reader.
- jasonpeacock 2y agoYou can also mount your git commits as a filesystem: https://jvns.ca/blog/2023/12/04/mounting-git-commits-as-folders-with-nfs/ https://jvns.ca/blog/2023/12/04/mounting-git-commits-as-fold... (previously: https://news.ycombinator.com/item?id=38527866 https://news.ycombinator.com/item?id=38527866 )
- amiga386 2y agoAll you need now is a giant pile of rules for which revisions to select and you have the unholy demon that is Rational ClearCase
- TheGlav 2y agoWhat!? you didn't add a versioned database layer on a server with code stored in clearcase that stored those ClearCase config specs to manage the configuration of your config specs to manage the configuration of your version control system that had your application configuration in it?! How did you even operate? /s
- skissane 2y agoI always thought Oracle ADE was a cooler demon. Shame the internal talk about productising it never went anywhere.
- Edmond 2y agoIf you're intrigued by this then Solvent-Configr might be of interest: https://aws.amazon.com/marketplace/pp/prodview-i3ym46leenag4 https://aws.amazon.com/marketplace/pp/prodview-i3ym46leenag4 It uses file system mechanics to model objects, meaning you can design object based solutions that support file system style navigation. Demo: https://youtu.be/XgTgubZQPHw https://youtu.be/XgTgubZQPHw
- sambeau 2y agoHa. I did this back in 2003. It's surprisingly fast, and makes it simple to do granular locking. I used it as a per-user database for a web-templating language for a giant web-site building tool.
- compressedgas 2y agoAlso the Parts-of-file File System: https://www.usenix.org/conference/2005-usenix-annual-technical-conference/parts-file-file-system https://www.usenix.org/conference/2005-usenix-annual-technic...
- ThinkBeat 2y agoNo XML, Excel,PDF or CSV support yet.
- IlliOnato 2y agoI guess support for XML would be tricky, because XML is just way more complex format than the ones already supported. It is still essentially a tree, but with additional structure. Representing elements and their contents is easy enough. But attributes, comments, processing instructions, entities... And remember, an XML document can include a DTD (it does not have to be in a separate file). To present it as a file system in a useful, non-convoluted way? I will be very, very interested if it's possible, but not holding my breath.
- eyelidlessness 2y agoOn the one hand, I can’t help but point out you forgot to mention the other big inherent complexity that would make XML-as-FS a uniquely complex beast: namespaces. On the other hand, I can’t help but point out that a related technology comes very close to demonstrating how you might map XML to a file system: XPath. Probably the biggest issue would be syntax, and again largely due to namespaces.
- RetroTechie 2y agoUseful enough that it should be an OS-level standard feature, imho. Unix-like OSes allow mounting disk images to explore their contents. But there's many more file formats where exploring files-inside-files is useful. Compressed archives, for one. Some file managers support those, but (imho) application-level is not the optimal layer to put this functionality. Could be implemented with a kind of driver-per-filetype.
- frizlab 2y agoHonest question: How is this useful? I don’t see any use-case where this would come in handy.
- TimeBearingDown 2y agoYou could seed compressed archives of massive text files or similar via BitTorrent while making the contents available to your apps in read-only mode.
- RetroTechie 2y agoIt allows you to use any tool available for regular files, on the files-in-files as well. As opposed to extract contents and then work on that (requiring extra steps + disk space). Or be limited to what specialized utilities support.
- russfink 2y agoExporting data to some format would be easy.
- crabbone 2y agoWe used this in Gitlab CI. Unfortunately, the only way they deal with artifacts is by putting them in Zip files. Cache between builds would thus be stored as a Zip file. However, fully extracting it before each build would sometimes take as much, if not more time than to just build fresh. Mounting a Zip file as a filesystem allows extracting entries on-demand, at the time a file access would've been made. This was a notable speedup in our compilation process.
- deleted 2y ago[deleted]
- chuckadams 2y agoNeat. Now how about a filesystem that takes a directory of files and exposes it as a single json file? You could call it the Filesystem File, and mount it in the File Filesystem if you wanted...
- paulgb 2y agoThis is really neat, but when I saw the headline I got excited that it was something I have been looking for / considering writing, and I figure the comments here would be a good place to ask if something like this exists: Is there a FUSE filesystem that runs in-memory (like tmpfs) while mounted, and then when dismounted it serializes to a single file on disk? The closest I can find are FUSE drivers that mount archive files, but then you don't get things like symlinks.
- khc 2y agodoes it have to be fuse? cant you mount a disk image with loopback
- metadat 2y agoBut will the disk image be fully stored in memory? No.. not with loopback. Either that, or it won't be mutable in memory with commit on unmount.
- generalizations 2y agoPut the disk image inside a ramdisk and it's in memory. Write a script for saving to physical disk when dismounting, and you're done.
- andrewflnr 2y agoWouldn't a disk image have a fixed size? It could be a pain to resize.
- ranger_danger 2y agoqcow2 is auto-expanding
- andrewflnr 2y agoBut not trivially loop-mountable. I guess it's possible, though. https://unix.stackexchange.com/questions/268460/how-to-mount-qcow2-image https://unix.stackexchange.com/questions/268460/how-to-mount...
- agumonkey 2y agoAll I see is a generic tree walk mechanism, here implemented in folders/files, but in plan9 it's .. plumber ?
- alephaleph 2y agoReminds me of Omar Rizwan's TabFS <https://omar.website/tabfs/ https://omar.website/tabfs/>
- secwang 2y agoremind me of djb's envdir
- waldrews 2y agoCould this be used in Windows by exposing a Samba file share from WSL or Docker?
- PMunch 2y agoOh this is cool! I recently wrapped libfuse in Nim and after porting the 'hello' filesystem example I made one which is more or less exactly this. However my version you pipe data and have to provide a mountpoint, then when it's done it writes the result over stdout. That means you can inline it in a pipe chain but also that you have to make sure to grab the output. At the moment I'm exploring other stuff which could be made into file systems. I've got a statusbar thing for the Nimdow window manager which allows you to write contents to individual files and it creates a bar with blocks on them as the output. It makes it super easy to swap out what is on your bar which is pretty neat. Another tool I've made is a music player. It uses libvlc and when given a folder it reads all the media with ID3 tags and sets up folders like 'by-artist', 'by-album', etc. Each file is named as '<track number> - <song title>' and contains the full path to the actual file. To play a song you cat one of these files into 'control/current' and write the word play to 'control/command'. There's a bit more to it like that like a playlist feature and some more commands, but that's the basic idea. The goal is to have a super-scriptable music player.
- lambdaxyzw 2y agoThis makes me think, it would be nice if there was an easy built-in way to expose information about a process using the filesystem. Something like "cat /proc/$pid/fs/current_track" to get a name of a current song from a music player, or "ls /proc/$pid/fs/tabs" to list open tabs in my browser (and maybe use this to grab the html or embedded images). I mean right now it's possible to do this using FUSE, but that's convoluted and nobody does it.
- PMunch 2y agoCan you actually write to the /proc directory? Working with libfuse I've had much of the same ideas, basically allow programs to expose information as files. The beauty of the fuse system is also that you only have to respond to requests when they happen. So you don't have to actually create all these files until someone asks for them. Another idea I've had is to expose the configuration of a program through a filesystem. Instead of having a config file and a refresh command or a complicated IPC you could simply write to files to change the config.
- purple-leafy 2y agoWhen I saw the title I thought it was a meme. But wow what a clever idea. Not sure id ever need to reach for it personally as I do most data processing in a higher level language, but I can imagine people can find use cases. Nice out of the box thinking
- timrobinson333 2y agoIt's an interesting idea but I think the usefulness would be greatly enhanced if it could handle json arrays; most needed json structures contain array elements in my experience
- MadcapJake 2y agoper an issue ticket[1], it can: setfattr -n user.type -v list # use xattr on macOS [1]: https://github.com/mgree/ffs/issues/66 https://github.com/mgree/ffs/issues/66
- planede 2y agoHmm, this opens the possibility to also commit these files as directory structures. I wonder how this would affect merges and conflicts.
- qazxcvbnm 2y agoWhat happens if your JSON key has a slash?
- sigmonsays 2y agothis is cool but it's fuse.. which is not so cool. these days i reach for jq.. I've recently became interested in duckdb too. Using a tool that is specialized for the format is usually more ideal than a generic one treating everything as files. There is a lot in JSON that can't be represented in a flexible way as files and directories. For instance, what if a key has "/" in it.. What happens to lists and their order when you re-serialize How are hashes represented.. how can you tell if a parent is a object or a list.. inserting a item into a list is a ton of error prone renames.. the list goes on (edited for formatting)
- willlma 2y agoThis reminds me of [pry](https://pry.github.io/ https://pry.github.io/) for Ruby that allows you to cd into other scopes