6 ms·

I would respectfully suggest that the author is misusing CI. If you have trouble running your tests locally, you have a problem. If you have trouble deploying f

by kyrofa 3y ago

I would respectfully suggest that the author is misusing CI. If you have trouble running your tests locally, you have a problem. If you have trouble deploying from your local code, you have a problem. All of those capabilities should exist as simple scripts in your project already. Once you have that done, the CI yaml is a simple glue layer that defines an order of operations, e.g.: 1. Run static tests 2. If those pass, run unit/integration tests 3. If those pass, deploy If you find yourself screaming about YAML, you're leaning too heavily on it and need to refactor your project's scripts. Maybe a good question to ask would be "if I had to switch to another CI system today, how hard would it be?" If the answer is "hard", perhaps you're leaning too heavily on it and need to refactor your project's scripts.

digitalsushi 3y ago
Testing locally, I agree with. Deployments, I think it'd be fair to consider the requirements. At work, our softare can be tested locally but deployments are all registered against a central authority, and after a point of composing enough access requirements, only then does a role (cicd in this case) have enough policy allowance to perform a deployment. The entire transaction is auditable. And I think that with a deployment, that's how it should be; allowing that trust down to a local environment strikes me that too much permission is accured with a single entity. I guess that we could better define what a deployment is; to some nonprod environments I'd agree, but I'd still probably insist on the heavy machinery up at the test, perf, qa, areas, and then getting into staging and prod, there'd be no wiggle room.
- kyrofa 3y ago
  Fair critique, totally depends on what kind of software we're talking about and where the deployment is happening. In general though, how screwed are you if your CI environment goes down? Can you not deploy anything? That would be scary. My point, however, was mostly that the logic necessary to deploy should live as part of your codebase, not written out in YAML. The privs necessary to deploy are a separate discussion.
msm_ 3y ago
This sounds fine and well, but it's not how Github Actions work (or at least, not the encouraged workflow). Let's have a look at the snippet from one of the projects I work on: - name: Set up Docker Buildx uses: docker/setup-buildx-action@v2 - name: Build and push mwdb-core image uses: docker/build-push-action@v4 with: file: ./deploy/docker/Dockerfile tags: | certpl/mwdb:${{ github.sha }} certpl/mwdb:master cache-from: | type=registry,ref=certpl/mwdb:buildcache outputs: type=docker,dest=./mwdb-image - name: Upload mwdb-core image uses: actions/upload-artifact@v3 with: name: mwdb-image path: mwdb-image Good luck running this locally. There's no script code to speak of, just references to external "actions" and parameters (for example, https://github.com/docker/setup-buildx-action https://github.com/docker/setup-buildx-action). Some CI platforms are just a simple glue layer (Gitlab CI - which I prefer - is one of them), but in most cases Github CI is not. Maybe it adds to the author frustration?
- jameshart 3y ago
  Building it that way is a choice. It's not mandatory. You can use gitlab CI with special-purpose docker images for all your steps and magic parameters driving everything too (Gitlab AutoDevops works that way). But if you just run your steps in shell scripts in vanilla docker images containing your build-time dependencies, you should be able to produce something that works the same in any CI pipeline, or locally. The most annoying thing for me is that a lot of CI engines make docker-in-docker complicated. I love using compose to set up integration test environments, but doing that in CI is often a fight.
  - kyrofa 3y ago
    > Building it that way is a choice. It's not mandatory. This ^ . In GitHub Actions, I personally try to use pre-baked actions as little as possible, for exactly the reasons I outlined. I prefer GitLab CI, but you can make a mess of that just as easily. In general, if you approach CI as I suggested, you end up with something maintainable regardless of the CI engine in use.
- javier2 3y ago
jameshart 3y ago
90% of the tricky parts of CI are secret management. As long as you write scripts to pick up credentials in a sane way (.awsprofile or similar) you should be able to configure your CI to provide the credentials just as well as you can locally - but in practice, the various different ways that things like artifact repositories, integration test databases, and cloud deployment tools want to manage auth is the cause of most of the complexity in getting your build/test/deploy pipeline working on the runner.
- twosdai 3y ago
  I would say the secret management part typically isn't the most insane or annoying part. For me it's external system state management. Like making sure the integration test db is cleaned up correctly.
djha-skin 3y ago
This is all fine and good if you are the principal developer of the project. However, the author makes it clear that he is migrating other people's CI pipelines. He is a DevOps engineer working across several teams. This is why he makes the important point that discipline is not enough. The reason is most teams simply don't care. I find one in five teams where everyone on the team cares about the build (when I'm lucky!), most teams have one person who cares, and some teams have no one that cares. When I am tasked with the proper care and feeding of the pipelines of others, I want tools that can work and help me out even when the developers who created the software are Holding it Wrong. With these requirements in mind -- managing and migrating the many different CI piplines across an organization -- it would be a major breakthrough to have a tool that 1) transpiles to the workflows of all the CI tools and 2) allows for local testing. So many orgs have different teams using different CI stacks, and the local testing problem is always a struggle. I would use a tool like that into the ground. So I would qualify your original statement: The author isn't misusing CI. Rather, the author is attempting to survive in a world where others are misusing it, and where the author is tasked with managing all the CI pipelines.
pyrale 3y ago
> I would respectfully suggest that the author is misusing CI. If you have trouble running your tests locally, you have a problem. I would respectfully suggest that you misread the author. The issue isn't running tests locally, it's running the CI config locally. I experienced the same problem with gitlab CI years ago, where, basically, you can lint the file and not much more. Past that, you need to run it through your CI and debug if you get slightly different results compared to running a script locally.
- capableweb 3y ago
  Yeah, it's a horrible experience all around. CircleCI solved this problem like a decade ago, enabling SSH builds so you can troubleshoot straight up in the build itself, and once you've figured it out, just copy-paste the steps to your Makefile/CI config. I don't understand how one could build a CI service so long time after CircleCI launched, and still not have that very same feature (or something similar).
hervem 3y ago
The authors speak about how hard it is nowadays to provide a CI extension to the major platform including GitHub, Azure DevOps and Gitlab. (and others) Wanting to run locally a developed extension is totally legit as some can be really tricky and depends on the behavior of [runner & OS].

by kyrofa 3y ago

digitalsushi 3y ago

Testing locally, I agree with. Deployments, I think it'd be fair to consider the requirements. At work, our softare can be tested locally but deployments are all registered against a central authority, and after a point of composing enough access requirements, only then does a role (cicd in this case) have enough policy allowance to perform a deployment. The entire transaction is auditable. And I think that with a deployment, that's how it should be; allowing that trust down to a local environment strikes me that too much permission is accured with a single entity. I guess that we could better define what a deployment is; to some nonprod environments I'd agree, but I'd still probably insist on the heavy machinery up at the test, perf, qa, areas, and then getting into staging and prod, there'd be no wiggle room.

kyrofa 3y ago
Fair critique, totally depends on what kind of software we're talking about and where the deployment is happening. In general though, how screwed are you if your CI environment goes down? Can you not deploy anything? That would be scary. My point, however, was mostly that the logic necessary to deploy should live as part of your codebase, not written out in YAML. The privs necessary to deploy are a separate discussion.

msm_ 3y ago

This sounds fine and well, but it's not how Github Actions work (or at least, not the encouraged workflow). Let's have a look at the snippet from one of the projects I work on: - name: Set up Docker Buildx uses: docker/setup-buildx-action@v2 - name: Build and push mwdb-core image uses: docker/build-push-action@v4 with: file: ./deploy/docker/Dockerfile tags: | certpl/mwdb:${{ github.sha }} certpl/mwdb:master cache-from: | type=registry,ref=certpl/mwdb:buildcache outputs: type=docker,dest=./mwdb-image - name: Upload mwdb-core image uses: actions/upload-artifact@v3 with: name: mwdb-image path: mwdb-image Good luck running this locally. There's no script code to speak of, just references to external "actions" and parameters (for example, https://github.com/docker/setup-buildx-action https://github.com/docker/setup-buildx-action). Some CI platforms are just a simple glue layer (Gitlab CI - which I prefer - is one of them), but in most cases Github CI is not. Maybe it adds to the author frustration?

jameshart 3y ago
Building it that way is a choice. It's not mandatory. You can use gitlab CI with special-purpose docker images for all your steps and magic parameters driving everything too (Gitlab AutoDevops works that way). But if you just run your steps in shell scripts in vanilla docker images containing your build-time dependencies, you should be able to produce something that works the same in any CI pipeline, or locally. The most annoying thing for me is that a lot of CI engines make docker-in-docker complicated. I love using compose to set up integration test environments, but doing that in CI is often a fight.
- kyrofa 3y ago
  > Building it that way is a choice. It's not mandatory. This ^ . In GitHub Actions, I personally try to use pre-baked actions as little as possible, for exactly the reasons I outlined. I prefer GitLab CI, but you can make a mess of that just as easily. In general, if you approach CI as I suggested, you end up with something maintainable regardless of the CI engine in use.
javier2 3y ago

jameshart 3y ago

90% of the tricky parts of CI are secret management. As long as you write scripts to pick up credentials in a sane way (.awsprofile or similar) you should be able to configure your CI to provide the credentials just as well as you can locally - but in practice, the various different ways that things like artifact repositories, integration test databases, and cloud deployment tools want to manage auth is the cause of most of the complexity in getting your build/test/deploy pipeline working on the runner.

twosdai 3y ago
I would say the secret management part typically isn't the most insane or annoying part. For me it's external system state management. Like making sure the integration test db is cleaned up correctly.

djha-skin 3y ago

This is all fine and good if you are the principal developer of the project. However, the author makes it clear that he is migrating other people's CI pipelines. He is a DevOps engineer working across several teams. This is why he makes the important point that discipline is not enough. The reason is most teams simply don't care. I find one in five teams where everyone on the team cares about the build (when I'm lucky!), most teams have one person who cares, and some teams have no one that cares. When I am tasked with the proper care and feeding of the pipelines of others, I want tools that can work and help me out even when the developers who created the software are Holding it Wrong. With these requirements in mind -- managing and migrating the many different CI piplines across an organization -- it would be a major breakthrough to have a tool that 1) transpiles to the workflows of all the CI tools and 2) allows for local testing. So many orgs have different teams using different CI stacks, and the local testing problem is always a struggle. I would use a tool like that into the ground. So I would qualify your original statement: The author isn't misusing CI. Rather, the author is attempting to survive in a world where others are misusing it, and where the author is tasked with managing all the CI pipelines.

pyrale 3y ago

> I would respectfully suggest that the author is misusing CI. If you have trouble running your tests locally, you have a problem. I would respectfully suggest that you misread the author. The issue isn't running tests locally, it's running the CI config locally. I experienced the same problem with gitlab CI years ago, where, basically, you can lint the file and not much more. Past that, you need to run it through your CI and debug if you get slightly different results compared to running a script locally.

capableweb 3y ago
Yeah, it's a horrible experience all around. CircleCI solved this problem like a decade ago, enabling SSH builds so you can troubleshoot straight up in the build itself, and once you've figured it out, just copy-paste the steps to your Makefile/CI config. I don't understand how one could build a CI service so long time after CircleCI launched, and still not have that very same feature (or something similar).

hervem 3y ago

The authors speak about how hard it is nowadays to provide a CI extension to the major platform including GitHub, Azure DevOps and Gitlab. (and others) Wanting to run locally a developed extension is totally legit as some can be really tricky and depends on the behavior of [runner & OS].