4 ms·
s3 just makes you work for it: - delete 1000 objects per post to /?delete - do this 100 times per connection - do this on a few threads per server to avoid b
by gfody 6y ago
s3 just makes you work for it:
- delete 1000 objects per post to /?delete
- do this 100 times per connection
- do this on a few threads per server to avoid being throttled
- which entails keeping track of which servers you're using (there are hundreds of servers available but dns only exposes a couple per second)
..and that assumes you've already got a list of keys - if you need to also enumerate the bucket you'll have to employ the same strategy and also partition your queries somehow to get multiple non-overlapping continuation-tokens for your concurrent calls to /?list. you can also request a bucket inventory but if you're in a hurry it's faster to do it yourself (for a 25TB bucket the turnaround on the inventory request is probably 2+ days)