3 ms·

> S3 requests to US buckets might be served by delayed replicas in other countries if the request comes from outside the US What? That makes no sense. Do you h

> S3 requests to US buckets might be served by delayed replicas in other countries if the request comes from outside the US What? That makes no sense. Do you have a source for that? I thought the explicit choice of region when creating a bucket limits where data is located. Why would the give you geo-replication for free? Also: "Amazon S3 creates buckets in a Region you specify. To optimize latency, minimize costs, or address regulatory requirements, choose any AWS Region that is geographically close to you." - https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.... Also the consistency when creating objects is described here: https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.... For new objects, you get read-after-write consistency and the example you gave contradicts that.

Dunedan 6y ago
> For new objects, you get read-after-write consistency and the example you gave contradicts that. Mind the documented caveat for this case: > The caveat is that if you make a HEAD or GET request to a key name before the object is created, then create the object shortly after that, a subsequent GET might not return the object due to eventual consistency.
- dividuum 6y ago
  Indeed, although that seems irrelevant for the mentioned use case. After all, why would the client request an image before it's uploaded. It seems like a straight forward "server side upload->generate url and send it to client->client requests image" flow. It's perfectly covered by the consistency provided.
  - fxtentacle 6y ago
    That's what we did. Server side upload, link into websocket, client downloads image. It did work 99.99% of all requests.
    - Dunedan 6y ago
      I'm surprised about the conclusion of the AWS support that this is intended behavior. I'm curious about the specifics of the behavior of S3 in this case. Do you have any more detailed information from the AWS support you could share by any chance? If you don't mind, I'd be also grateful if you could drop some answers to the questions below. Do you use S3 replication (CRR/SRR)? Before uploading, do you make any request to the object key? If yes, what kind of request? For downloading, do you use pre-signed URLs or is the S3 bucket public? What's the error the 0,01% of users encounter? Is it a NoSuchKey error?
fxtentacle 6y ago
They also say "A process writes a new object to Amazon S3 and immediately lists keys within its bucket. Until the change is fully propagated, the object might not appear in the list" which implies that you cannot read immediately after write.
- dividuum 6y ago
  No. It does not imply that. It only implies that it might not be included in a List Operation. That's all this sentence tells you.

Dunedan 6y ago

> For new objects, you get read-after-write consistency and the example you gave contradicts that. Mind the documented caveat for this case: > The caveat is that if you make a HEAD or GET request to a key name before the object is created, then create the object shortly after that, a subsequent GET might not return the object due to eventual consistency.

dividuum 6y ago
Indeed, although that seems irrelevant for the mentioned use case. After all, why would the client request an image before it's uploaded. It seems like a straight forward "server side upload->generate url and send it to client->client requests image" flow. It's perfectly covered by the consistency provided.
- fxtentacle 6y ago
  That's what we did. Server side upload, link into websocket, client downloads image. It did work 99.99% of all requests.
  - Dunedan 6y ago
    I'm surprised about the conclusion of the AWS support that this is intended behavior. I'm curious about the specifics of the behavior of S3 in this case. Do you have any more detailed information from the AWS support you could share by any chance? If you don't mind, I'd be also grateful if you could drop some answers to the questions below. Do you use S3 replication (CRR/SRR)? Before uploading, do you make any request to the object key? If yes, what kind of request? For downloading, do you use pre-signed URLs or is the S3 bucket public? What's the error the 0,01% of users encounter? Is it a NoSuchKey error?

fxtentacle 6y ago

They also say "A process writes a new object to Amazon S3 and immediately lists keys within its bucket. Until the change is fully propagated, the object might not appear in the list" which implies that you cannot read immediately after write.

dividuum 6y ago
No. It does not imply that. It only implies that it might not be included in a List Operation. That's all this sentence tells you.