3 ms·

Is there not a better solution for this? We have this baked into our docker builds and it irks me that we have to copy personal credentials into a docker build

by gbear0 5y ago

Is there not a better solution for this? We have this baked into our docker builds and it irks me that we have to copy personal credentials into a docker build so we can use git to pull modules and build. Is everyone actually doing this, or do you setup read-only tokens per private module, or anything else?

morelisp 5y ago
We set up an internal Athens[0] caching proxy with credentials to pull from our GitLab. So far it seems to be win/win/win - We don't need this dance in our Dockerfiles / build scripts / dev machines anymore. We have a baked GOPROXY / GONOSUMDB in our build image and developers configure the same proxy (via `go env -w`) locally. - We pull all packages over the proxy, so builds are faster / use less bandwidth / still mostly work when GitHub goes down. - SCAs get more difficult; the credentials the proxy has are more limited than any individual developer's GitLab tokens / credentials, and owning the proxy is going to be harder than a single developer's laptop. [0] https://docs.gomods.io/ https://docs.gomods.io/
ramonrue 5y ago
At least for Gitlab, you can write a .netrc in the user‘s home dir like so: machine <your GitLab domain> (e.g. gitlab.com) login <your GitLab id> password <your GitLab personal access token> For the access token, you can also leverage Gitlab‘s CI Job Token. What we do is an „echo $(netrc contents with $CI_JOB_TOKEN) > ~/.netrc“ in the pre-script in CI. (https://stackoverflow.com/a/61257782 https://stackoverflow.com/a/61257782 and https://docs.gitlab.com/ee/security/token_overview.html https://docs.gitlab.com/ee/security/token_overview.html)
- morelisp 5y ago
  If you do take this approach, writing your access token in plaintext into a well-known file in your home directory seems like a strictly worse idea than the (also more broadly-compatible) Git insteadof rule to pull via SSH and your local agent.
  - ramonrue 5y ago
    Yes, though it should be noted that the access token is only valid for the duration of the Job.
BigGreenTurtle 5y ago
What I've done on a recent project is to vendor the dependencies before build(which are excluded from git, you could cache these if you're using a CICD system like circle), copy everything into the build container, build with -mod=vendor and then copy the resulting binary into a new container. Don't know how that'll work out for a larger project but it works well enough in this small API I wrote.
travisd 5y ago
For Go, I also take the vendor approach. But for other stuff (namely private npm packages), (relatively) modern versions of Docker support build-time-only secrets: https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information https://docs.docker.com/develop/develop-images/build_enhance.... Pass the secret as part of the docker build command and then access it inside RUN --mount=... stages.

by gbear0 5y ago

morelisp 5y ago

We set up an internal Athens[0] caching proxy with credentials to pull from our GitLab. So far it seems to be win/win/win - We don't need this dance in our Dockerfiles / build scripts / dev machines anymore. We have a baked GOPROXY / GONOSUMDB in our build image and developers configure the same proxy (via `go env -w`) locally. - We pull all packages over the proxy, so builds are faster / use less bandwidth / still mostly work when GitHub goes down. - SCAs get more difficult; the credentials the proxy has are more limited than any individual developer's GitLab tokens / credentials, and owning the proxy is going to be harder than a single developer's laptop. [0] https://docs.gomods.io/ https://docs.gomods.io/

ramonrue 5y ago

At least for Gitlab, you can write a .netrc in the user‘s home dir like so: machine <your GitLab domain> (e.g. gitlab.com) login <your GitLab id> password <your GitLab personal access token> For the access token, you can also leverage Gitlab‘s CI Job Token. What we do is an „echo $(netrc contents with $CI_JOB_TOKEN) > ~/.netrc“ in the pre-script in CI. (https://stackoverflow.com/a/61257782 https://stackoverflow.com/a/61257782 and https://docs.gitlab.com/ee/security/token_overview.html https://docs.gitlab.com/ee/security/token_overview.html)

morelisp 5y ago
If you do take this approach, writing your access token in plaintext into a well-known file in your home directory seems like a strictly worse idea than the (also more broadly-compatible) Git insteadof rule to pull via SSH and your local agent.
- ramonrue 5y ago
  Yes, though it should be noted that the access token is only valid for the duration of the Job.

BigGreenTurtle 5y ago

What I've done on a recent project is to vendor the dependencies before build(which are excluded from git, you could cache these if you're using a CICD system like circle), copy everything into the build container, build with -mod=vendor and then copy the resulting binary into a new container. Don't know how that'll work out for a larger project but it works well enough in this small API I wrote.

travisd 5y ago

For Go, I also take the vendor approach. But for other stuff (namely private npm packages), (relatively) modern versions of Docker support build-time-only secrets: https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information https://docs.docker.com/develop/develop-images/build_enhance.... Pass the secret as part of the docker build command and then access it inside RUN --mount=... stages.