4 ms·
Would this enable atomic operations with S3 (e.g. "put object only if it does not exist")?
by mholt 6y ago
Would this enable atomic operations with S3 (e.g. "put object only if it does not exist")?
- OJFord 6y agoCould you not already do that by specifying ETag?
- rcaught 6y agoNo, because the object may not be there (but already uploaded) with eventual consistency.
- sterwill 6y agoBefore this update, S3 provided "read-after-write" consistency for object creation at a unique path[0] with PUT and subsequent GET operations. Replacement wasn't consistent. So I think you could have worked out some kind of atomic pattern with S3 objects, using new keys for each state change. [0] https://github.com/awsdocs/amazon-s3-developer-guide/blob/41df5c1f948b64e26d5f2583d4f97d47568a7baa/doc_source/Introduction.md#ConsistencyModel https://github.com/awsdocs/amazon-s3-developer-guide/blob/41...
- scarface74 6y agoThe Etag is not a reliable hash of the file contents. It will be different if the file was uploaded in multiple parts like using the CLI than if it was moved as one operation like copying from one bucket to another. You can tell the difference because the Etag will have a “-“ in it if it was uploaded as a multipart upload.
- ryanworl 6y agoI also want the answer to this, but to the best of my knowledge it does not.
- leakybucket 6y agoThe copy object operation has options to control if the copy occurs based on examining source metadata, see "x-amz-copy-source-if-match" at https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObje... . With the consistency change, those might be useful as the basis for atomic operations.