4 ms·

CI for me has to access repositories. Eg. downloading libraries from PyPI, Maven, NPM, etc... For private repositories, that means access to credentials. Proba

CI for me has to access repositories. Eg. downloading libraries from PyPI, Maven, NPM, etc... For private repositories, that means access to credentials. Probably read-only credentials, but it requires network access. Would you be suggesting that everyone should commit all dependencies?

formerly_proven 5y ago
That's a fair point. I think for many projects which only consume dependencies the answer can be "yes, just commit all your dependencies" in many instances. Pulling from internal repos shouldn't be critical though as long as the read-only tokens you're using can only access what the developers can read anyway; pulling "unintended" code in a CI should never be able to escalate because we're running anything a dev pushed.
- NAHWheatCracker 5y ago
  I've heard of doing this, although not worked on such a project. Committing node_modules seems wild to me and I foresee issues with committing all the Python wheels necessary for different platforms on some projects. I'm a proponent of lock-file approaches, which gain 99% of the benefits with far less pain. It requires network access, though.
  - emteycz 5y ago
    Yarn v2 is much better for committing dependencies than committing node_modules. However, it's problematic. Only use it if you're certain it will solve a specific problem you have. Consider using Docker to build images that include a snapshot of node_modules.
nightpool 5y ago
Sure, but that's a completely different scenario than the one the OP/TFA is talking about. We're talking about disclosing privileged information or access tokens that a single developer on their own shouldn't be able to handle without code review, like being able to use your CI system to access production accounts. Software libraries don't fall under this category, since the developer would already need to have accessed them to write the code in the first place!
- NAHWheatCracker 5y ago
  I was focused on the "no network access" aspect of formerly_proven's comment. "no network access" would make CI nearly pointless from my perspective. I agree that there are tokens and variables that are dangerous to expose via CI, but throwing the baby out with the bathwater confused me.
derefr 5y ago
You could "just" 1. stand up a private package mirror that you control, that uses a whitelist for what packages it is willing to mirror; 2. configure your project's dependency-fetching logic to fetch from said mirror; 3. configure CI to only allow outbound network access to your package mirror's IP. The disadvantage — but also the point — of this, is that it is then a release manager's responsibility, not a developer's responsibility, to give the final say-so for adding a dependency to the project (because only the release manager has the permissions to add packages to the mirror.)
- NAHWheatCracker 5y ago
  That's an interesting approach. I'm not keen on the bureaucratic aspect, which leads to more friction than it's worth in my experience. I guess that's beside the point if your goal is only to reduce risk of compromised CI/CD.
  - mulmen 5y ago
    You can compromise and give the developers the ability to add dependencies as well. In a separate “dependency mirror” package. But those have to be code reviewed like anything else. So you have a paper trail but adding a dependency is lower friction. And you still can’t accidentally (or maliciously!) pull in an evil dependency without at least two people involved.

formerly_proven 5y ago

That's a fair point. I think for many projects which only consume dependencies the answer can be "yes, just commit all your dependencies" in many instances. Pulling from internal repos shouldn't be critical though as long as the read-only tokens you're using can only access what the developers can read anyway; pulling "unintended" code in a CI should never be able to escalate because we're running anything a dev pushed.

NAHWheatCracker 5y ago
I've heard of doing this, although not worked on such a project. Committing node_modules seems wild to me and I foresee issues with committing all the Python wheels necessary for different platforms on some projects. I'm a proponent of lock-file approaches, which gain 99% of the benefits with far less pain. It requires network access, though.
- emteycz 5y ago
  Yarn v2 is much better for committing dependencies than committing node_modules. However, it's problematic. Only use it if you're certain it will solve a specific problem you have. Consider using Docker to build images that include a snapshot of node_modules.

nightpool 5y ago

Sure, but that's a completely different scenario than the one the OP/TFA is talking about. We're talking about disclosing privileged information or access tokens that a single developer on their own shouldn't be able to handle without code review, like being able to use your CI system to access production accounts. Software libraries don't fall under this category, since the developer would already need to have accessed them to write the code in the first place!

NAHWheatCracker 5y ago
I was focused on the "no network access" aspect of formerly_proven's comment. "no network access" would make CI nearly pointless from my perspective. I agree that there are tokens and variables that are dangerous to expose via CI, but throwing the baby out with the bathwater confused me.

derefr 5y ago

You could "just" 1. stand up a private package mirror that you control, that uses a whitelist for what packages it is willing to mirror; 2. configure your project's dependency-fetching logic to fetch from said mirror; 3. configure CI to only allow outbound network access to your package mirror's IP. The disadvantage — but also the point — of this, is that it is then a release manager's responsibility, not a developer's responsibility, to give the final say-so for adding a dependency to the project (because only the release manager has the permissions to add packages to the mirror.)

NAHWheatCracker 5y ago
That's an interesting approach. I'm not keen on the bureaucratic aspect, which leads to more friction than it's worth in my experience. I guess that's beside the point if your goal is only to reduce risk of compromised CI/CD.
- mulmen 5y ago
  You can compromise and give the developers the ability to add dependencies as well. In a separate “dependency mirror” package. But those have to be code reviewed like anything else. So you have a paper trail but adding a dependency is lower friction. And you still can’t accidentally (or maliciously!) pull in an evil dependency without at least two people involved.