3 ms·
Partial writes would need to buffered and flush()ed. That is not unlike other file systems, until you fully know it is persisted - or am I missing something?
by sebazzz 4y ago
Partial writes would need to buffered and flush()ed. That is not unlike other file systems, until you fully know it is persisted - or am I missing something?
- tyingq 4y agoThat you can't update an S3 object. There is no equivalent to open() on an existing file, seek() to some position, and modify a few bytes.
- rbanffy 4y agoThat S3 storage is almost immediately consistent on a planetary scale is pretty impressive already. This would make it a lot more complicated to do that. OTOH, you can always engineer around such limitations by storing smaller objects you can completely ingest and store on a single operation.
- stingraycharles 4y agoIt’s not supported by S3 to partially update an object; you would have to completely replace the entire object. While they could do some magic under the hood to make it seem like it works, it’s better to just reject these operations and let the application developer decide what’s best.
- bilekas 4y agoMaybe I'm missing some context here, but can't you generate X number of presigned URL's with your chunk size and if you provide the chunk ordering it's automatically done on S3 side, but I though you could manually implement the chunk order assignment also.
- ldoughty 4y agoI would suspect the team making this tool is not exactly the same team maintaining S3... So adding "server side operations" would require work from the other team to be added.. which certainly could be done if they decided it was needed. As a first MVP release this project seems neat for read use cases.. and feedback from the community can help ensure read operations are solid while they are also working on the write side -- which they admitted was lacking in this initial release, but want to improve.