4 ms·
> literally millions of downloads per month I was surprised it would be so low given all the Dockerfiles building in CI systems everywhere and apparently from
by marmaduke 6y ago
> literally millions of downloads per month
I was surprised it would be so low given all the Dockerfiles building in CI systems everywhere and apparently from
https://pypistats.org/top https://pypistats.org/top
It looks like tens of millions per day a billions per month
edit which is ~115 downloads per second, which is nothing a beefy VM couldn’t handle.
- dmw_ng 6y agoBigQuery: #standardSQL SELECT EXTRACT(HOUR FROM timestamp) hour, COUNT(*) AS num_downloads FROM `the-psf.pypi.file_downloads` WHERE DATE(timestamp) BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) AND CURRENT_DATE() GROUP BY hour ORDER BY num_downloads Gives us 6.956 billion downloads in the past 30 days With hour 15 clocking in at around 2,964 requests/sec -- mean. This is still not representative of peak instantaneous traffic, which I'd hazard a guess is somewhere closer to 10x that number. Now factor in the 'usual' case where the CDN absorbs 70% of this, and you're still left with a service that is non-trivial to run and support 24/7. PyPi manages to only have a few small downtimes per year.
- marmaduke 6y agoHadn’t thought to look up more precise numbers, and the comment on CDN is good one. It’s an example of where smart choices in the client (pip) could keep the backend (PyPI) scalable.