4 ms·

I found it took a while to get to the point. In the end, I actually agree with this! I have also been thinking about filesystems, which are trees of of diction

I found it took a while to get to the point. In the end, I actually agree with this! I have also been thinking about filesystems, which are trees of of dictionaries (directories) and blobs (files), and that there are many other examples of tree-like data. Data structures in our programs are tree shaped, perhaps with references/pointers to other parts of the tree. JSON is a tree of dictionaries (objects), arrays, and data (the primitive string/number/null). The arrays are ordered, much like the content inside a HTML/XML block. I agree that adding an "array" style of directory to our file systems would be really cool. I've been toying with the idea of writing a FUSE driver that holds some structured data (possibly including arrays) and just converts the (integer) index into a string. The idea is that you could e.g. view and edit some JSON tree with the file explorer. And not just JSON - basically any piece of structured data that we have in our programs can be "viewed" as a FS this way (e.g. just convert structs into directories of fields). It could even be a pretty cool and universal debugger - a breakpoint could make the program pause and serve a FUSE driver and later continue when it is unmounted :) And yes, exactly what follows from this is some program could be written to open some "directory" and render a "document" based on the contents. The filesystem supports links, so we have the "web" like experience. The "document" angle does require adding a kind of directory with ordered array semantics rather than dictionary/map semantics. It's the first missing ingredient listed in the article. Though some filesystems use sorted dictionaries (b-trees or whatever) for directory maps so you could maybe hack this ordered semantics in that way. The second missing ingredient listed in the article is the hypermedia part. I mean my computer is actually OK at inferring if a file is a movie or photo or text document, so we kind of have a way of dealing with that, too. The "blob of bytes is a narrow waste" thing is quite powerful. That said, sum types could be useful to demark different kinds of "stuff", and there's no reason an implementation of this idea couldn't support sum types as part of its data model.

pilgrim0 2y ago
> I found it took a while to get to the point. You're just a great understandeour > I agree that adding an "array" style of directory to our file systems would be really cool. I think this is sort of a low-hanging fruit people have slept on. We've proved list-based systems are extremely versatile for data structures (s-expr) and programming (lisps). What about for media in general? Everywhere I look I just see lists, with very minor stylistic distinctions between them. Of course there're abysmal infrastructural differences between chats, feeds and what not, but it does not invalidate a universal list-based frontend, similar to what you developed in your comment.
- andyferris 2y ago
  True. I didn't really think about (linked) lists much, only (flat) arrays, but maybe that's possible already with files laid out like: /head /next/head /next/next/head ... etc or whatever. I'm not saying its _ergonomic_, mind you. I'd like my file viewer to lay out the list flat, for example. There might be technical limitains with the length of the file path. Etc.
- syberant 2y ago
  I'd argue that this concept for media is most commonly known as "playlists" and unfortunately only used within data silos, e.g. a playlist of YouTube videos, a Spotify playlist, a TV season of episodes, a series of episodes, a trilogy, etc. (Yes, I'd argue that your music library of mp3 files is also kind of a silo, although portable) Heck, even a slideshow is arguably a playlist. I agree that putting a playlist-like concept into, say, the filesystem would be an extremely interesting idea but I think a big danger is running into the same problem as hardlinks and symlinks. This problem is that if a file is "present" in multiple places (or playlists) deleting/modifying/moving it can have unforeseen consequences and it's hard to reason about (and if you copy the file now you get to invent a way to track different versions too!). I think this is also holding tagging filesystems back. I'm currently writing a non-hierarchical FUSE filesystem and have been thinking about this list-directory concept but I'm still not completely sure how it would work, especially since I need to remain backwards compatible with the POSIX interfaces. Will probably have to just try it out (xattrs to the rescue?) and see what sticks I suppose... A linkdump of interesting somewhat related stuff: - https://newsletter.squishy.computer/p/knowledge-structures https://newsletter.squishy.computer/p/knowledge-structures - https://newsletter.squishy.computer/p/all-you-need-is-links https://newsletter.squishy.computer/p/all-you-need-is-links - https://thesephist.com/posts/search-vs-nav/ https://thesephist.com/posts/search-vs-nav/ - https://karl-voit.at/2017/02/10/evolution-of-systems/ https://karl-voit.at/2017/02/10/evolution-of-systems/ Especially "Information-Centric Systems" - https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/ https://www.theatlantic.com/magazine/archive/1945/07/as-we-m... A classic 1945 article cited as inspiration by Ted Nelson, Doug Engelbart and Tim Berners-Lee. - https://en.wikipedia.org/wiki/Content-addressable_storage https://en.wikipedia.org/wiki/Content-addressable_storage - https://www.nayuki.io/page/designing-better-file-organization-around-tags-not-hierarchies https://www.nayuki.io/page/designing-better-file-organizatio...
andyferris 2y ago
I should probably have mentioned that there already exists FUSE-based JSON FS mounters, for example this one is 13 years old: https://github.com/calebcase/jsonfs https://github.com/calebcase/jsonfs

pilgrim0 2y ago

> I found it took a while to get to the point. You're just a great understandeour > I agree that adding an "array" style of directory to our file systems would be really cool. I think this is sort of a low-hanging fruit people have slept on. We've proved list-based systems are extremely versatile for data structures (s-expr) and programming (lisps). What about for media in general? Everywhere I look I just see lists, with very minor stylistic distinctions between them. Of course there're abysmal infrastructural differences between chats, feeds and what not, but it does not invalidate a universal list-based frontend, similar to what you developed in your comment.

andyferris 2y ago
True. I didn't really think about (linked) lists much, only (flat) arrays, but maybe that's possible already with files laid out like: /head /next/head /next/next/head ... etc or whatever. I'm not saying its _ergonomic_, mind you. I'd like my file viewer to lay out the list flat, for example. There might be technical limitains with the length of the file path. Etc.
syberant 2y ago
I'd argue that this concept for media is most commonly known as "playlists" and unfortunately only used within data silos, e.g. a playlist of YouTube videos, a Spotify playlist, a TV season of episodes, a series of episodes, a trilogy, etc. (Yes, I'd argue that your music library of mp3 files is also kind of a silo, although portable) Heck, even a slideshow is arguably a playlist. I agree that putting a playlist-like concept into, say, the filesystem would be an extremely interesting idea but I think a big danger is running into the same problem as hardlinks and symlinks. This problem is that if a file is "present" in multiple places (or playlists) deleting/modifying/moving it can have unforeseen consequences and it's hard to reason about (and if you copy the file now you get to invent a way to track different versions too!). I think this is also holding tagging filesystems back. I'm currently writing a non-hierarchical FUSE filesystem and have been thinking about this list-directory concept but I'm still not completely sure how it would work, especially since I need to remain backwards compatible with the POSIX interfaces. Will probably have to just try it out (xattrs to the rescue?) and see what sticks I suppose... A linkdump of interesting somewhat related stuff: - https://newsletter.squishy.computer/p/knowledge-structures https://newsletter.squishy.computer/p/knowledge-structures - https://newsletter.squishy.computer/p/all-you-need-is-links https://newsletter.squishy.computer/p/all-you-need-is-links - https://thesephist.com/posts/search-vs-nav/ https://thesephist.com/posts/search-vs-nav/ - https://karl-voit.at/2017/02/10/evolution-of-systems/ https://karl-voit.at/2017/02/10/evolution-of-systems/ Especially "Information-Centric Systems" - https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/ https://www.theatlantic.com/magazine/archive/1945/07/as-we-m... A classic 1945 article cited as inspiration by Ted Nelson, Doug Engelbart and Tim Berners-Lee. - https://en.wikipedia.org/wiki/Content-addressable_storage https://en.wikipedia.org/wiki/Content-addressable_storage - https://www.nayuki.io/page/designing-better-file-organization-around-tags-not-hierarchies https://www.nayuki.io/page/designing-better-file-organizatio...

andyferris 2y ago

I should probably have mentioned that there already exists FUSE-based JSON FS mounters, for example this one is 13 years old: https://github.com/calebcase/jsonfs https://github.com/calebcase/jsonfs