16 ms·
Not OP but we also had to move a relatively large S3 bucket (significantly larger than 25TB) and unfortunately AWS doesn't have a way to change the ownership of
by dmlittle 6y ago
Not OP but we also had to move a relatively large S3 bucket (significantly larger than 25TB) and unfortunately AWS doesn't have a way to change the ownership of the S3 bucket. My guess is it has to do with how the underlying system stores the objects.
We ended up writing a Go program to copy the objects from one bucket to another to help with the parallelization of migration. This was also before AWS had announced S3 Batch operations so I'm not sure how much better it would be to use that today. The deletion of the bucket also took us over a week. Even though we had deleted all objects in the bucket due to the eventual consistency nature of S3 we weren't allowed to delete the bucket until all objects were fully removed from S3. All we could get from AWS support was to wait a few more days and reach back if we couldn't delete it then.
Edit: depending on your object naming scheme you might also run into the S3 prefix rate limits.