8 ms·
This is good thinking, and mostly overlaps with what I've tried to do a few times, so I would love to see it finally happen somehow. For example, the Newton st
by wrs 9y ago
This is good thinking, and mostly overlaps with what I've tried to do a few times, so I would love to see it finally happen somehow.
For example, the Newton storage system I worked on at Apple 1990-1996 was based on separating organization from storage so we could have multiple (tag and/or hierarchy) organization systems. That eventually became the "soup" system in the shipping Newton OS, where objects were retrieved by content rather than hierarchy.
More recently, I spent some painful years working on Microsoft's WinFS, which had a ton of overlap with the principles here, and demonstrates just how hard it is to go from some nice principles that all seem like the Right Thing to an actual successful adopted implementation of the principles.
- discreteevent 9y agoWould you have time to say a bit more about why WinFS failed from your point of view?
- wrs 9y agoOh man, that would take a book. If I had to severely boil it down: You can have a beautiful unifying idea that seems to make total sense in the abstract, but then bog down completely when you have to make real implementation decisions in the context of a massive existing platform with a several-thousand-person development team. At some point a system's assumptions become so established that there are certain aspects that are literally impossible to change. Good intentions, even with the full support of the highest levels of management, can't resolve all the tangles.
- nayuki 9y agoThere is a series of blog posts that gives a retrospective on WinFS: * https://hal2020.com/2013/02/14/winfs-integratedunified-storage-and-microsoft-part-1/ https://hal2020.com/2013/02/14/winfs-integratedunified-stora... * https://hal2020.com/2013/02/14/winfs-integratedunified-storage-and-microsoft-part-2/ https://hal2020.com/2013/02/14/winfs-integratedunified-stora... * https://hal2020.com/2013/02/25/winfs-integratedunified-storage-and-microsoft-part-3/ https://hal2020.com/2013/02/25/winfs-integratedunified-stora... * https://hal2020.com/2013/03/10/winfs-integratedunified-storage-and-microsoft-part-4/ https://hal2020.com/2013/03/10/winfs-integratedunified-stora...
- wrs 9y agoWow, I forgot Hal practically did wrote a book. :) I was on the Windows Shell side, sitting in a lot of schema meetings where products couldn't figure out how to combine their incompatible complex existing schemas into one new WinFS schema that would be understandable to anyone, and in a lot of other meetings where I was sort of a negotiation translator, decoding user interface concepts for the SQL team and SQL concepts for the Shell team, though their priorities were wildly different. The common theme throughout was that it was pretty clear none of the product teams saw any reason they should go through unknown years of effort and compatibility hell, delaying the feature roadmap for their product, just to fulfill Bill's desire for this abstract notion of Integrated Storage. So these meetings went on interminably to make Bill and David and Bob and Jim happy, while none of the participants outside the WinFS team itself saw any point to the exercise. It was not a pleasant experience, but I learned a lot about how a large organization works! (A reason I'm no longer in one.)
- yiyus 9y agoI think most people would agree a tag filesystem or a similar concept is a great idea. I have wanted one myself for a long time. Yet, for some reason, it doesn't take off. Do you think it is just a problem of implementation?
- Baeocystin 9y agoHierarchical file systems allow you to say for sure where a file isn't. Every tagging system I've ever played with has turned out to be a mess in actual use compared to the simplicity a hierarchy provides. (I am just a user, not a developer of filesystems, so this may be a naive opinion.)
- kijin 9y agoTags are fun when you have a few thousand items to test your MVP with. It gets much less fun when you have millions of items with thousands of tags, all on a flat hierarchy. On the other hand, when you're stuck with a flat hierarchy anyway (e.g. thousands of pictures, all named DCIMxxxx.jpg), tags can be more useful. But only if they're automatic. I want the best of both worlds. I want to organize my stuff into folders and use tags to search for individual items. There's no need to be a purist on either side. "Designing better file organization around tags" is a good thing. "Designing better file organization around tags, not hierarchies" is not.
- nayuki 9y agoI understand your concerns, and they are indeed valid. First off, I doubt that managing millions of files in a traditional hierarchical file system is fun either. You'd likely run into problems with making unique names, sharding folders, and categorizing files that logically belong in multiple places. I also have some worries that existing file systems (say NTFS or XFS) will behave or perform well with millions of files. I believe that implementing tags is a starting point for the problem of managing millions of files in a sensible way. Speaking of thousand of pictures, what I really want is to dump all my photos into one folder. Right now, tools are ill-equipped to deal with large folders, so I am forced to manually create a new folder for every thousand or so items.
- Angostura 9y agoIf you were one of the people behind the ideas in Soup, I salute you.
- wrs 9y agoThanks! That was pretty much just me, but it was inspired by reading a whole bunch of other peoples' books and papers on persistent object systems. That was a big topic of research at the time. I don't really know why the persistent object idea fell out of sight so completely.