3 ms·
I feel the same about dependency steps in CI, without a cache or any similar structure. Package repos like Rubygems, NPM and PyPi get utterly rinsed by the cont
by ljm 5y ago
I feel the same about dependency steps in CI, without a cache or any similar structure. Package repos like Rubygems, NPM and PyPi get utterly rinsed by the continual downloading and redownloading of stuff the client should have already stored.
- trulyme 5y agoThis. And both with GitHub and GitLab it takes quite a bit of an extra effort to setup caching. It hurts to see 'npm ci' download half the internet every time a developer pushes to dev server.
- yellow_lead 5y agoWould be interesting to speculate about the greenhouse effect of all these repeated downloads
- bombcar 5y agoIt used to be possible to have a squid or even a backwards varnish cache handle lots of this but https everything has made that much harder to do. Still possible, however.
- ljm 5y agoIf nothing else, it is patently wasteful and, as a user, you don't really see CI billed in terms of network bandwidth. Just indirectly through the equivalent of mainframe minutes. And even then, that's not enough to discourage anyone from building a suboptimal pipeline.
- Seirdy 5y agoThat's why the first step of CI for me, when possible, is to rsync a .tar.gz file from the server I'm deploying to. The tarball contains statically-linked binaries and other stuff I'll need for the build. It's also a good reason for CI providers to mirror package repositories.
- SilverRed 5y agoNothing is going to change until the hosters make it a pain to abuse. Rubygems could require an api key to download from and rate limit that key. Sure you could attempt to generate a bunch of keys and cycle them but it would be easier to just cache your gems.