4 ms·
I always wondered by never bothered to implement: What if I would implement an S3-backed block storage. Like every 16MB chunk is stored in a separate object. A
by vbezhenar 3y ago
I always wondered by never bothered to implement:
What if I would implement an S3-backed block storage. Like every 16MB chunk is stored in a separate object. And filesystem driver would download/upload chunks just like it does it with HDD.
And then format this block device with Ext4 or use it in any other way (LVM, encrypted FS, RAID and so on).
Is it usable concept?
- candiodari 3y agohttps://github.com/s3fs-fuse/s3fs-fuse/wiki/Fuse-Over-Amazon https://github.com/s3fs-fuse/s3fs-fuse/wiki/Fuse-Over-Amazon Seems like a usable concept yes.
- cbluth 3y agoYes, its called s3backer
- vbezhenar 3y agoThanks, that's the thing.
- damagednoob 3y agoNot sure if it's a usable concept but it sounds similar to what someone else did[1]: > ...I implemented a virtual file system that fetches chunks of the database with HTTP Range requests when SQLite tries to read from the filesystem... [1]: https://phiresky.github.io/blog/2021/hosting-sqlite-databases-on-github-pages/ https://phiresky.github.io/blog/2021/hosting-sqlite-database...
- kirubakaran 3y agohttps://github.com/s3fs-fuse/s3fs-fuse https://github.com/s3fs-fuse/s3fs-fuse
- vbezhenar 3y agoIt uses S3 as a filesystem. I'm talking about using S3 as a block device. Slightly different approach.
- franky47 3y agoSounds like a very expensive way to build a filesystem.
- DaiPlusPlus 3y agoWhat you're describing already exists in Azure Blob Storage (Azure's S3 compete), known as Page Blobs, which is also how you host VM HDDs (VHDs) in Azure, as it allows for page-level (i.e. disk-block-level) reads/writes. I'm not clued-in to AWS's equivalent, but a quick google suggests it's AWS EBS (Elastic Block Storage), though it doesn't seem to coexist side-by-side with normal S3 storage, and their API for reading/writing blocks looks considerably more complicated than Azure's: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-accessing-snapshot.html https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-acce...
- yread 3y agoTheir fuse-based filesystem driver even supported CACHED block access ie you would download, cache and read the blocks you needed locally instead of sending every read over the wire. Close to local fs performance for little space
- DaiPlusPlus 3y agoHow does that handle concurrent writes to the same block? How is the cache invalidated?
- mappu 3y agoThe single FUSE process can coordinate all that with an in-memory mutex. This thread is in the context of filesystems like ext4, which are completely unsafe for concurrent mounts - if you connect from multiple machines then it will just corrupt horrifically. If your filesystem was glusterfs, moose/lizard, or lustre, then they have ways to handle this (with on-disk locks/semaphores in some other sector).
- electroly 3y agoBehind the scenes, EBS Snapshots are stored in S3. When you first create an EBS volume from a snapshot, it pulls blocks from S3 on-demand.
- bob1029 3y ago