26 ms·
The S3 API doesn't have the concurrency control primitives necessary to guarantee consistency in the face of concurrent writes. I wrote a distributed lock on G
by FooBarWidget 3y ago
The S3 API doesn't have the concurrency control primitives necessary to guarantee consistency in the face of concurrent writes.
I wrote a distributed lock on Google Cloud Storage. https://www.joyfulbikeshedding.com/blog/2021-05-19-robust-distributed-locking-algorithm-based-on-google-cloud-storage.html https://www.joyfulbikeshedding.com/blog/2021-05-19-robust-di...
During my research it was quickly evident that GCS has more concurrency control primitives than S3. Heck S3 didn't even guarantee strong read-after-write until recently.
- fs111 3y ago> until recently Almost 3 years https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/ https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-rea...
- janejeon 3y agoAh, I'm guessing that's the same reason why many "cloud storage as a backend" stuff (e.g. Terraform/Vault storage backends, Delta.io backends) require DynamodDB + S3
- electroly 3y agoWhile I'm sure you're right, this VFS implementation doesn't use any of that stuff. They just handwave concurrent writes away by saying the application has to ensure a single writer on its own. An S3 implementation appears to be planned; it's mentioned in the comments in the bcvmodule.c file.