4 ms·
As someone who has spent a lot of time on this problem: you don't want to store tags as attributes/resource forks/streams/etc in a traditional filesystem. The s
by drbawb 8y ago
As someone who has spent a lot of time on this problem: you don't want to store tags as attributes/resource forks/streams/etc in a traditional filesystem. The single most important characteristic for a tag-based organization system is that search operations must be extremely fast and cheap. Walking a hierarchical filesystem and stat'ing a bunch of files to discover their tag information is anything but cheap.
What you want is to store the tag metadata heap in an easily indexed structure (i.e: a search tree.) That heap then just contains pointers into content addressable storage. You can kind of build such a thing on top of existing filesystems which support hardlinks, however I'd advise against that for the metadata itself. Once you have the power of tagged content a natural evolution is querying that data using set operators. You will inevitably re-invent (some approximation of) SQL once you arrive at this discovery.
- fxfan 8y agoThat's a great point. Which rules out using alternate streams as a tagging system. But not tags themselves- so I'll still like a file-system that supports tags natively.
- the8472 8y agoIt might still make sense to keep the canonical metadata in xattrs so that tools can interoperate on it and build a search index from it when you actually need to do search. On a NVMe drive rebuilding the search index shouldn't be too costly, even on hundred thousands of files. I think there were also some ioctl additions for btrfs that let you sequentially walk the filesystem by extents or inodes instead of by directory tree, which makes indexing a lot easier.