4 ms·
S3 is not even files, and definitely not a filesystem. The thing I would expect from a file abstraction is mutability. I should be able to edit pieces of a fi
by alphazard 3y ago
S3 is not even files, and definitely not a filesystem.
The thing I would expect from a file abstraction is mutability. I should be able to edit pieces of a file, grow it, shrink it, read and write at random offsets. I shouldn't have to go back up to the root, or a higher level concept once I have the file in hand. S3 provides a mutable listing of immutable objects, if I want to do any of the mutability business, I need to make a copy and re-upload. As originally conceived, the file abstraction finds some sectors on disk, and presents them to the client as a contiguous buffer. S3 solves a different problem.
Many people misinterpret the Good Idea from UNIX "everything is a file" to mean that everything should look like a contiguous virtual buffer. That's not what the real Good Idea is. Really: everything can be listed in a directory, including directories. There will be base leaves, which could be files, or any object the system wants to present to a process, and there will be recursive trees (which are directories). The directories are what make the filesystem, not the type of a particular leaf. Adding a new type of leaf, like a socket or a frame buffer, or whatever, is almost boring, and doesn't erode the integrity of the real good idea. Adding a different kind of container like a list, would make the structure of the filesystem more complex, and that would erode the conceptual integrity.
S3 doesn't do any of these things, and that's fine. I just want a place to put things that won't fit in the database, and know they won't bitrot when I'm not looking. The desire to make S3 look more like a filesystem comes from client misunderstanding of what it's good at/for, and poor product management indulging that misunderstanding instead of guarding the system from it.
- akerl_ 3y agoHow do read-only filesystems align with your definition?
- yencabulator 3y agoYou can't create new things on a read-only filesystem, you can in S3; not a good analogy.
- akerl_ 3y agoI wasn’t making an analogy. I was asking how read-only filesystem works given the parent commenters description of what makes something a filesystem.
- yencabulator 3y agoIt's a filesystem where many operations return an error (historically, EROFS). There are many things you can't do with one. Is that interesting somehow? I don't agree with defining a filesystem as something that has to be backed by a block device, but the shape of a filesystem API is historically very different from the shape of the S3 API.
- paulddraper 3y agoAll of the read stuff still applies (list, open, read, seek).
- mr_toad 3y agoYou can read individual blocks on a read only file system. With S3 you’re stuck with range requests which are much larger.
- thinkharderdev 3y ago> S3 is not even files, and definitely not a filesystem. I agree. To me the correct analog for S3 is a block storage device (a very weird one where blocks can be any size and can have a key associated with them) and not a filesystem. A filesystem is an abstraction that sits on top of a block storage device and so an "S3 filesystem" would have to be an abstraction that sits on top of S3 as the underlying block storage.
- paulddraper 3y ago> a very weird one where blocks can be any size and can have a key associated with them That is a very weird one