3 ms·
Pip, maven, npm, docker hub, apt-get, github and suchlike seem to manage to provide hefty binary downloads with no need to monetise anonymous downloads. I don'
by michaelt 3y ago
Pip, maven, npm, docker hub, apt-get, github and suchlike seem to manage to provide hefty binary downloads with no need to monetise anonymous downloads.
I don't quite understand how they do so, but it seems to be possible. Everyone who downloads a 5GB stable diffusion model probably downloaded 5GB of pytorch+cuda+cudnn first.
- Lord_Zero 3y agoBack in the day, universities would host all that shit via their mirrors for free for everyone. Not sure where it all is today with the cloud being so prevalent.
- zo1 3y agoYou don't even hear the term mirror anymore.
- basilgohar 3y agoLinux distros still have large networks of mirrors for downloading their releases and also packaged binary updates. Maybe we don't "hear" the term but they are still there.
- outside1234 3y agoWe call them CDNs now :)
- brucethemoose2 3y ago> 5GB of pytorch+cuda+cudnn This itself is kinda insane. In fact, it was literally untenable for PyPi, which is why (historically) installing all that was unecessarily painful.
- 0xcde4c3db 3y agoFor its part, PyPI is apparently funded by the Python Software Foundation and run by volunteer admins, with the bulk of its data served gratis by Fastly. This is based on a blog post about its infrastructure from 2021 [1], which also says that the list price for the throughput they're doing is $1.8 million per month. [1] https://dustingram.com/articles/2021/04/14/powering-the-python-package-index-in-2021/ https://dustingram.com/articles/2021/04/14/powering-the-pyth...
- bpiche 3y agoSpacy hosts them too, of course. python -m spacy download en-core-web-lg etc