4 ms·

>I'd like to go the other way, and have a filesystem that leverages a proper database for its metadata. Microsoft made an attempt back in 2003 at creating a "s

by jasode 3y ago

>I'd like to go the other way, and have a filesystem that leverages a proper database for its metadata. Microsoft made an attempt back in 2003 at creating a "smarter" file system with rich metadata based on a real database engine (MS SQL Server) ... but eventually abandoned the project: https://en.wikipedia.org/wiki/WinFS https://en.wikipedia.org/wiki/WinFS WinFS wasn't necessarily going to completely replace NTFS but Microsoft did have ambitious plans for its integration into everything. It's interesting that other platforms like Unix/Linux/Apple that were designing "next generation" file systems like ZFS/ext4/APFS after the failed WinFS project had a chance to try adding their own WinFS smart metadata features to but they didn't. The later fragmentation of user data being spread around in the cloud such as AWS objects, Apple iCloud, Backblaze backups, etc instead of every file saved on home computers makes a "universal rich metadata index stored in a database engine" to be a cross-platform standard by all operating systems less realistic today than back in 2003. Therefore, the core os file systems remain a "dumb blob of bytes" with a separate layer of metadata index db in vendor-specific schemes like Apple Spotlight db, Windows Search in ".edb" file, etc.

rkagerer 3y ago
Yeah they totally missed the mark with WinFS, in my opinion. It was too exotic/foreign with no clear adoption path. All I want is dumb files with something like SQLite replacing or augmenting the allocation table. Then sprinkle on some new capabilities incrementally.
didgetmaster 3y ago
When WinFS was canceled, I started my own pet project to build a new kind of file system that totally changed the way file metadata was stored and managed. I built an object based system that could easily handle hundreds of millions of files within a single container (what I called a pod) and do searches much like how you would search a large database table. The 'file table' contains a 64 byte record for each object and has some very unique characteristics that make searches incredibly fast (e.g. it can find all the photos in under a second, even if there are 20 million of them among 200 million files). You can also attach various metadata 'tags' to each object and find matching objects based on them. The project is in open beta and anyone can download it and try it out on their own computer. https://www.didgets.com https://www.didgets.com

by jasode 3y ago

rkagerer 3y ago

Yeah they totally missed the mark with WinFS, in my opinion. It was too exotic/foreign with no clear adoption path. All I want is dumb files with something like SQLite replacing or augmenting the allocation table. Then sprinkle on some new capabilities incrementally.

didgetmaster 3y ago

When WinFS was canceled, I started my own pet project to build a new kind of file system that totally changed the way file metadata was stored and managed. I built an object based system that could easily handle hundreds of millions of files within a single container (what I called a pod) and do searches much like how you would search a large database table. The 'file table' contains a 64 byte record for each object and has some very unique characteristics that make searches incredibly fast (e.g. it can find all the photos in under a second, even if there are 20 million of them among 200 million files). You can also attach various metadata 'tags' to each object and find matching objects based on them. The project is in open beta and anyone can download it and try it out on their own computer. https://www.didgets.com https://www.didgets.com