> it's nowhere near reading/writing speeds of e.g. a single 600MB file. I have to disagree here. If you look at benchmarks on internet, yes, it will look like

> it's nowhere near reading/writing speeds of e.g. a single 600MB file. I have to disagree here. If you look at benchmarks on internet, yes, it will look like S3 is dead slow. But that is a client problem, not an S3 problem. For instance, boto3 (s3transfer) is an awful implementation that was so overengineered with a reimplementation of futures, task stealing, etc, that the download throughput is pathetic. Most often it will make you top below 200MB/s. But S3 itself scales very well if you know how to use it, and skip boto3. From my experience and benchmarks, each S3 connection will deliver up to 80MB/s, and with range requests you can easily have multiple parallel blocks downloaded. I wrote a simple library that does this called s3pd (https://github.com/NewbiZ/s3pd https://github.com/NewbiZ/s3pd). It's not perfect and is process based instead of coroutines, but that will give you an idea. For reference, using s3pd I can saturate any EC2 network interface that I found (tested up to 10Gb/s interfaces, with download speed >1GB/s). Boto is really doing bad press to S3.