3 ms·

This looks great. One question: We're currently using Drone (the open-source self-hosted version), which allows us to define a number of parallel sidecar contai

This looks great. One question: We're currently using Drone (the open-source self-hosted version), which allows us to define a number of parallel sidecar containers ("compose") as part of the build pipeline. We start things like PostgreSQL and Elasticsearch like this because a lot of tests need to exercise the actual data storage instead of (or in addition to) mocking/emulating them. The container builder thus doubles as a CI server, since every build needs to be tested anyway, and tests need to go through very much the same steps (install dependencies, build source and so on). Tests benefit from being run on the exact same filesystem that the final code runs on, guaranteeing that dependencies are identical. From what I can tell, such a setup is not supported by GCCB?

skj 10y ago
My general advice is to do integration testing after push. You say that you value the testing harness having the same set of dependencies as your deployable artifact. With GCCB, you are not limited to one image with your build - it can be any number of them. So, something like 1. build the deployable as gcr.io/project/service:${REVISION_ID} 2. build the testing harness as gcr.io/project/testing:${REVISION_ID} 3. build the db container as gcr.io/project/db:${REVISION_ID} Then, you can spin up a test environment using the ${REVISION_ID} tags, and do your integration testing there with the same nice guarantees. That all said, it's certainly possible to do fancier things within the build itself. Using the gcr.io/cloud-builders/docker build step, you can run any docker command. We haven't seen the need to directly support a docker-compose build step, but making it would be easy (different entrypoint on the docker build step) and it would automatically connect to the worker host's daemon. You can also run several of these docker build steps in parallel, or run one that does 'docker run --name=foo -d ...' and have the others run with '--link foo:foo', etc. We try not to limit the sorts of things you can do, and provide bonus features like the docker daemon's socket, service account credentials etc. You don't even need to build a container image, if you don't want to. Just provide some container images and we'll run them for you, pipe the logs back to you, and let you know how it went.
- lobster_johnson 10y ago
  Hm, that's a bit much. In particular as various apps have a different setup (we're a microservice shop and have tons of them, although a minority needs external dependencies for tests like these). Splitting it up like that also generates a dependency, which adds a burden on an already complicated workflow. Drone just solves that so much better by keeping everything in a single, simple build file.
tonglil 10y ago
I agree, the semantics required to define the build in GCCB seems really limiting, going to stick to Drone until this matures a bit. The plugin ability of Drone really outshines the current as well.

skj 10y ago

My general advice is to do integration testing after push. You say that you value the testing harness having the same set of dependencies as your deployable artifact. With GCCB, you are not limited to one image with your build - it can be any number of them. So, something like 1. build the deployable as gcr.io/project/service:${REVISION_ID} 2. build the testing harness as gcr.io/project/testing:${REVISION_ID} 3. build the db container as gcr.io/project/db:${REVISION_ID} Then, you can spin up a test environment using the ${REVISION_ID} tags, and do your integration testing there with the same nice guarantees. That all said, it's certainly possible to do fancier things within the build itself. Using the gcr.io/cloud-builders/docker build step, you can run any docker command. We haven't seen the need to directly support a docker-compose build step, but making it would be easy (different entrypoint on the docker build step) and it would automatically connect to the worker host's daemon. You can also run several of these docker build steps in parallel, or run one that does 'docker run --name=foo -d ...' and have the others run with '--link foo:foo', etc. We try not to limit the sorts of things you can do, and provide bonus features like the docker daemon's socket, service account credentials etc. You don't even need to build a container image, if you don't want to. Just provide some container images and we'll run them for you, pipe the logs back to you, and let you know how it went.

lobster_johnson 10y ago
Hm, that's a bit much. In particular as various apps have a different setup (we're a microservice shop and have tons of them, although a minority needs external dependencies for tests like these). Splitting it up like that also generates a dependency, which adds a burden on an already complicated workflow. Drone just solves that so much better by keeping everything in a single, simple build file.

tonglil 10y ago

I agree, the semantics required to define the build in GCCB seems really limiting, going to stick to Drone until this matures a bit. The plugin ability of Drone really outshines the current as well.