4 ms·
BigQuery: #standardSQL SELECT EXTRACT(HOUR FROM timestamp) hour, COUNT(*) AS num_downloads FROM `the-psf.pypi.file_downloads` WHERE DATE(timest
by dmw_ng 6y ago
BigQuery:
#standardSQL
SELECT EXTRACT(HOUR FROM timestamp) hour, COUNT(*) AS num_downloads
FROM `the-psf.pypi.file_downloads`
WHERE DATE(timestamp)
BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
AND CURRENT_DATE()
GROUP BY hour
ORDER BY num_downloads
Gives us 6.956 billion downloads in the past 30 days
With hour 15 clocking in at around 2,964 requests/sec -- mean. This is still not representative of peak instantaneous traffic, which I'd hazard a guess is somewhere closer to 10x that number.
Now factor in the 'usual' case where the CDN absorbs 70% of this, and you're still left with a service that is non-trivial to run and support 24/7. PyPi manages to only have a few small downtimes per year.
- marmaduke 6y agoHadn’t thought to look up more precise numbers, and the comment on CDN is good one. It’s an example of where smart choices in the client (pip) could keep the backend (PyPI) scalable.