4 ms·

Coincidentally, I came to this fresh from watching "Database as Filesystem"[1,2], which takes a different view: > Most network file systems are either a layer

Coincidentally, I came to this fresh from watching "Database as Filesystem"[1,2], which takes a different view: > Most network file systems are either a layer over an existing filesystem (NFS, CIFS), or are develped from scratch to have separate, replicated, purpose-designed databases for metadata and object store (GFS, Glusterfs). At the same time, most database engines provide (or can be coerced into providing) replication and all the ACID properties needed for a high-performance filesystem. > Idea: Use a database engine (Postgres, MariaDB) on raw partitions with a fast separate nVME log file; build POSIX file system semantics on top. It's pretty obvious that this could work; I'm just starting to implement it so performance and durability can be measured. Epic fail fans and optimists alike will be excited by an actual implementation attempt, I guess :) 1: https://www.youtube.com/watch?v=wN6IwNriwHc https://www.youtube.com/watch?v=wN6IwNriwHc 2: https://2019.linux.conf.au/schedule/presentation/212/ https://2019.linux.conf.au/schedule/presentation/212/

Felz 8y ago
S3QL does this with sqlite. An actual external DB would give you multiple users at once, though. I've been using a combination of S3/Postgres to get filelike semantics myself, but not with a POSIX interface. The biggest problem for general use is that S3 is really for immutable file storage, and not so much things that'll change a lot. That can be worked around, obviously.
1337shadow 8y ago
Thanks for sharing this ! it's lovely from a technical perspective and researching with Peter Chubb sounds like a transcending experience ... but if you have Bob the manager instead of Peter Chubb, and that you intend to practice continuous delivery on your product codebase, making backups part of your automated deployment pipeline introduces an ever growing bottleneck.

Felz 8y ago

S3QL does this with sqlite. An actual external DB would give you multiple users at once, though. I've been using a combination of S3/Postgres to get filelike semantics myself, but not with a POSIX interface. The biggest problem for general use is that S3 is really for immutable file storage, and not so much things that'll change a lot. That can be worked around, obviously.

1337shadow 8y ago

Thanks for sharing this ! it's lovely from a technical perspective and researching with Peter Chubb sounds like a transcending experience ... but if you have Bob the manager instead of Peter Chubb, and that you intend to practice continuous delivery on your product codebase, making backups part of your automated deployment pipeline introduces an ever growing bottleneck.