2 ms·
That's what my small python package does (https://github.com/NewbiZ/s3pd https://github.com/NewbiZ/s3pd). Definitely not perfect (not saturating cores because n
by Galanwe 6y ago
That's what my small python package does (https://github.com/NewbiZ/s3pd https://github.com/NewbiZ/s3pd). Definitely not perfect (not saturating cores because no use of event loop per process) but download is split in multiple processes and stored in shared memory.
I've been able to saturate 20GB NICs on Ec2 with it (32 cores)