3 ms·

Frankly, we’ve been fantasizing of moving away from the file-system-as-tree abstraction since forever, wanting to embrace a metadata rich database approach. Ca

by eecc 7y ago

Frankly, we’ve been fantasizing of moving away from the file-system-as-tree abstraction since forever, wanting to embrace a metadata rich database approach. Can’t happen soon enough imho

jjoonathan 7y ago
But we're not moving towards a metadata rich database, we're moving towards balkanized proprietary walled gardens in the cloud.
- eecc 7y ago
  That's true. The problem is that even when you pay for service, the incentive is to suck your data away, data-mine the hell out of it and hold you at ransom. It's like cable TV, originally one paid a premium to avoid advertisements. Today it's just the norm, and you get the advertisements anyway.
- kccqzy 7y ago
  Both of these are true. We're moving towards walled gardens in the cloud, but within these cloud, your "files" are becoming rows in a highly sophisticated database, perhaps in addition to blobs in a highly sophisticated object storage system.
  - TeMPOraL 7y ago
    But it doesn't matter because you - the user - aren't exposed to the benefits or even the abstraction of that database. You don't get to make arbitrary queries for metadata. You get access to whatever half-featured interface the vendor bothered to implement, whereas the database is optimized for their operational and datamining purposes.
bcyn 7y ago
> a metadata rich database approach Can you ELI5 what this means? Very curious to understand ideas on alternatives to the traditional file-system abstraction.
- nekopa 7y ago
  Not the guy you were asking but... I do a lot of self-study, and I use Calibre (ebook library software) to organise my reading materials. I used it because my Linux file browser has problems displaying covers for EPUB files. (And a couple of other) I like that my reading material has lots of ways for me to organise it without creating duplicates (tags mainly) search and browse my books. Then one evening I found myself wishing I could have all my other relevant 'files' in the same system: audio, video, source code, repos, blogs, podcasts, notes etc... Basically a metadata rich database - except not really a database in my case as I need to pull in 'stuff' from all over the place, and not only on one device. Files are nice, and I hate anything that takes away my ability to organise stuff on a physical level (which, truth be told Calibre stubbornly refuses to let you set up your folder system, so I cheat by using tags) but at the same time, the problem with the skeuomorphic paradigm of files and folder is that there are so many more options open in the digital world than in the real world when dealing with items. Imagine being able to sit at your office desk and open the drawer to pull out a piece of paper. When you look at the paper it shows you some text. Rotate the paper to the left and the text disappears and a video starts playing, turn it to the right and you hear a podcast, place it on your desk and 4 more pieces of paper appear next to it - or it calls someone relevant to the info. All possible with digital info, but not with a real piece of paper - yet. That is the only thing I see as a real negative about files and folders - it may be limiting what we can really do with our digital world because we're stuck in a file-system metaphor created hundreds, if not thousands, of years ago.
- kaslai 7y ago
  Typically when people talk about that, they mean they want to move from an approach where files have a location to an approach where files are queried based on metadata. So rather than going to /home/kaslai/images/ to (hopefully) see all my images, I'd instead query my filesystem, perhaps like /owner:kaslai/type:image. The specific syntax would need some serious thought, especially if we want to make it fairly ergonomic to slot in to existing systems. There are a large number of approaches that one could take. I'm personally a fan of a hybrid, where you could navigate to a location and then make queries on anything within that location. IMO one of the coolest features that such a filesystem could have is dealing with music. There's no perfect way to map CD rips to a hierarchical filesystem, but that wouldn't be a problem if I could just open a "directory" like /home/kaslai/music/?artist/?album/ which would automatically make virtual directories based on the artist and album, or I could just use /home/kaslai/music?artist=Smash\ Mouth to get a list of all 182 copies of All Star that I have in my library.
flukus 7y ago
We're definitely moving away from file-system-as-tree, but our destination seems to be more like grandma saving every file on the desktop than anything resembling rich metadata.
goatlover 7y ago
Wasn't that the way Google Docs was originally implemented, but everyone preferred having it organized more like a file system, so Google Drive has a folder structure.
erichanson 7y ago
Agree! Can't happen soon enough. My project Aquameta is one such attempt, trying to rethink the programming stack, and by extension all user data, as relational data instead of files, with the filesystem still there under the hood, but basically as a bootstrapping mechanism to get the database running. Here's an essay about the reasoning and approach: http://blog.aquameta.com/intro-chpater2-filesystem/ http://blog.aquameta.com/intro-chpater2-filesystem/ The TLDR is that the filesystem lacks a sophisticated information model, and doesn't provide mechanisms for defining structures, and we really need to embrace this basic requirement at a systemic level. Boot to PostgreSQL! :)
z3t4 7y ago
Almost all file systems have metadata. Its up to the apps what metadata to store and display. Music files have artist, album etc. photos have location, camera settings etc. The tree structure is just a feature. You can for example make one file appear in many folders. User rights are often linked to the tree structure, which is very useful.

by eecc 7y ago

Frankly, we’ve been fantasizing of moving away from the file-system-as-tree abstraction since forever, wanting to embrace a metadata rich database approach. Can’t happen soon enough imho

jjoonathan 7y ago

But we're not moving towards a metadata rich database, we're moving towards balkanized proprietary walled gardens in the cloud.

eecc 7y ago
That's true. The problem is that even when you pay for service, the incentive is to suck your data away, data-mine the hell out of it and hold you at ransom. It's like cable TV, originally one paid a premium to avoid advertisements. Today it's just the norm, and you get the advertisements anyway.
kccqzy 7y ago
Both of these are true. We're moving towards walled gardens in the cloud, but within these cloud, your "files" are becoming rows in a highly sophisticated database, perhaps in addition to blobs in a highly sophisticated object storage system.
- TeMPOraL 7y ago
  But it doesn't matter because you - the user - aren't exposed to the benefits or even the abstraction of that database. You don't get to make arbitrary queries for metadata. You get access to whatever half-featured interface the vendor bothered to implement, whereas the database is optimized for their operational and datamining purposes.

bcyn 7y ago

> a metadata rich database approach Can you ELI5 what this means? Very curious to understand ideas on alternatives to the traditional file-system abstraction.

nekopa 7y ago
Not the guy you were asking but... I do a lot of self-study, and I use Calibre (ebook library software) to organise my reading materials. I used it because my Linux file browser has problems displaying covers for EPUB files. (And a couple of other) I like that my reading material has lots of ways for me to organise it without creating duplicates (tags mainly) search and browse my books. Then one evening I found myself wishing I could have all my other relevant 'files' in the same system: audio, video, source code, repos, blogs, podcasts, notes etc... Basically a metadata rich database - except not really a database in my case as I need to pull in 'stuff' from all over the place, and not only on one device. Files are nice, and I hate anything that takes away my ability to organise stuff on a physical level (which, truth be told Calibre stubbornly refuses to let you set up your folder system, so I cheat by using tags) but at the same time, the problem with the skeuomorphic paradigm of files and folder is that there are so many more options open in the digital world than in the real world when dealing with items. Imagine being able to sit at your office desk and open the drawer to pull out a piece of paper. When you look at the paper it shows you some text. Rotate the paper to the left and the text disappears and a video starts playing, turn it to the right and you hear a podcast, place it on your desk and 4 more pieces of paper appear next to it - or it calls someone relevant to the info. All possible with digital info, but not with a real piece of paper - yet. That is the only thing I see as a real negative about files and folders - it may be limiting what we can really do with our digital world because we're stuck in a file-system metaphor created hundreds, if not thousands, of years ago.
kaslai 7y ago
Typically when people talk about that, they mean they want to move from an approach where files have a location to an approach where files are queried based on metadata. So rather than going to /home/kaslai/images/ to (hopefully) see all my images, I'd instead query my filesystem, perhaps like /owner:kaslai/type:image. The specific syntax would need some serious thought, especially if we want to make it fairly ergonomic to slot in to existing systems. There are a large number of approaches that one could take. I'm personally a fan of a hybrid, where you could navigate to a location and then make queries on anything within that location. IMO one of the coolest features that such a filesystem could have is dealing with music. There's no perfect way to map CD rips to a hierarchical filesystem, but that wouldn't be a problem if I could just open a "directory" like /home/kaslai/music/?artist/?album/ which would automatically make virtual directories based on the artist and album, or I could just use /home/kaslai/music?artist=Smash\ Mouth to get a list of all 182 copies of All Star that I have in my library.

flukus 7y ago

We're definitely moving away from file-system-as-tree, but our destination seems to be more like grandma saving every file on the desktop than anything resembling rich metadata.

goatlover 7y ago

Wasn't that the way Google Docs was originally implemented, but everyone preferred having it organized more like a file system, so Google Drive has a folder structure.

erichanson 7y ago

Agree! Can't happen soon enough. My project Aquameta is one such attempt, trying to rethink the programming stack, and by extension all user data, as relational data instead of files, with the filesystem still there under the hood, but basically as a bootstrapping mechanism to get the database running. Here's an essay about the reasoning and approach: http://blog.aquameta.com/intro-chpater2-filesystem/ http://blog.aquameta.com/intro-chpater2-filesystem/ The TLDR is that the filesystem lacks a sophisticated information model, and doesn't provide mechanisms for defining structures, and we really need to embrace this basic requirement at a systemic level. Boot to PostgreSQL! :)

z3t4 7y ago

Almost all file systems have metadata. Its up to the apps what metadata to store and display. Music files have artist, album etc. photos have location, camera settings etc. The tree structure is just a feature. You can for example make one file appear in many folders. User rights are often linked to the tree structure, which is very useful.