3 ms·
Isn't most of the bandwidth spent on packages? Even immediately after the initial install you probably have gigabytes of updates + new packages.
by mishafb 5y ago
Isn't most of the bandwidth spent on packages? Even immediately after the initial install you probably have gigabytes of updates + new packages.
- systemvoltage 5y agoI often wonder - who pays for pip packages that millions of server apps and CI/CD pipelines pull? There is some caching but still it’s gotta be enormous.
- dwrodri 5y agoThe official PyPI packkages are hosted on Fastly. I believe they donate the infrastructure costs through the Python Software Foundation, as the bandwidth is over 300TB a day, and would cost about 1.8 million dollars a month. Source: https://dustingram.com/articles/2021/04/14/powering-the-python-package-index-in-2021/ https://dustingram.com/articles/2021/04/14/powering-the-pyth...
- minusf 5y agointeresting article. where is the 300 TB number coming from? > The “maximum” here is per-day, so this means that at the peak (which was the day I made these graphs) PyPI served nearly 900 terabytes over more than 2 billion requests per day.
- dwrodri 5y agoThis tweet from Dustin Ingram: https://twitter.com/di_codes/status/1235707819955032069 https://twitter.com/di_codes/status/1235707819955032069 I think this number is dated but its what I stumbled across.
- minusf 5y agomight be. either way the numbers are impressive although i am afraid this massive bandwidth is necessary because of the docker's and ci/cd's that don't use caches...
- yjftsjthsd-h 5y agoYeah, it still blows my mind that nobody seems to have solved putting packages in torrents.
- rovr138 5y agoApt-torrent - https://directory.fsf.org/wiki/Apt-torrent https://directory.fsf.org/wiki/Apt-torrent I haven't used this a long time, but it used to be faster than mirrors where I was.