3 ms·
I find it's easier and more convenient to use aws-cli [1] (with subprocess.run) than using Boto. Aws-cli is multithreaded in both download and upload. There is
by ggregoire 6y ago
I find it's easier and more convenient to use aws-cli [1] (with subprocess.run) than using Boto.
Aws-cli is multithreaded in both download and upload. There is even a setting to tweak the number of parallel requests [2].
[1] https://github.com/aws/aws-cli https://github.com/aws/aws-cli
[2] https://docs.aws.amazon.com/credref/latest/refdocs/setting-s3-max_concurrent_requests.html https://docs.aws.amazon.com/credref/latest/refdocs/setting-s...
Edit: didn't know about s5cmd (from other commenters). Seems like a faster alternative to aws-cli.