3 ms·
Yes, and sometimes even listing can take days. I worked somewhere that a person decided using Twitter Firehose was a good idea for S3. Keyed by tweet per file
by base698 5y ago
Yes, and sometimes even listing can take days.
I worked somewhere that a person decided using Twitter Firehose was a good idea for S3. Keyed by tweet per file.
Ended up figuring out a way to get them in batches and condense. Ended up costing about $800 per hour to fix coupled with lifecycle changes they mentioned.
- orf 5y ago> Yes, and sometimes even listing can take days. You have a versioned bucket with a lot of delete markers in it. Make sure you've got a lifecycle policy to clean them up.
- properdine 5y agoDoing an S3 object inventory can be a lifesaver here!