6 ms·

I fail to see the point being made in the article. CI done right is the best thing that ever happened to me in terms of collaborative productivity. Of course t

I fail to see the point being made in the article. CI done right is the best thing that ever happened to me in terms of collaborative productivity. Of course there are many pitfalls. Like with any tool it becomes trivial to use it wrong. Integrating less than once per day and deferring all testing and linting to the CI process is an anti-pattern and the issue is not the CI process but how the team chooses to use it. For example I worked in a team of 10 and we were doing multiple production deploys per day. This was made possible by a great CI workflow. Everyone was running tests precommit and focusing on keeping the pipeline green. Yes I've seen the opposite as well, but that usually is a symptom of issues with the team not the general concept of CI. Not saying that there isn't room for improvement but all the real-time collaboration features I used on low-code platforms feel like a step back.

benjiweber 4y ago
I think the point is about far shorter feedback loops than possible with today's pervasive tech. > something even more continuous than continuous integration. It might look like real-time integration of your changes, collaborative editing, or smarter build systems that produce near-live feedback e.g. https://medium.com/darklang/how-dark-deploys-code-in-50ms-771c6dd60671 https://medium.com/darklang/how-dark-deploys-code-in-50ms-77...
- eterps 4y ago
  Yes exactly, I was also thinking about how darklang does it. The feedback loop can be much shorter, and I'm sure it's not just limited to how darklang does it either.
- cntainer 4y ago
  Well, how short the feedback loop needs to be is something that can be controlled by the process, and the team sets the process. I can have the normal flow, which takes minutes and runs all the tests and other stuff I might have on my pipeline (security, performance, etc). I use this for typical day to day work on features. I don't care if the deploy takes 50ms or 5 minutes. I can have a fast-track for critical production patches. I skip all the main CI steps and just get my code quickly in production. If done right this takes seconds. I didn't know about dark but I've seen this type of promise too many times, so I'm pretty sure there are many tradeoffs hidden under the nice shiny exterior. I can't know until I try it, but that kind of complexity doesn't all just disappear, i always has a cost even if it's out of sight.
  - jka 4y ago
    Here's the thing: we have hundreds of thousands of teams and organizations who currently each need to select and define those processes. Perhaps some of those teams and individuals are experts at determining the correct, minimal continuous integration suite to run on a per-commit basis to minimize time and energy expenditure without compromising correctness. But I can guarantee that not all (in fact, not many) are, and that they pay maintenance and mental overheads to adhere to those practices. It feels to me like there is a potentially massive opportunity to design an integrated language-and-infrastructure environment that only re-runs necessary checks based on each code diff. - "Altered a single-line docstring in a Python file? OK, the data-flow from that infers a rebuild of one of our documentation pages is required, let's do that... done in 2ms" - "Refactored the variable names and formatting within a C++ class without affecting any of the logic or ABI? OK, binary compatibility and test compatibility verified as unchanged, that's a no-op... done in 0ms" - "Renamed the API spec so that the 'Referer' header is renamed 'Referrer'? OK, that's going to invalidate approximately a million downstream server and client implementations, would you like to proceed?" (examples are arbitrary and should imply no specific limitations or characteristics of languages or protocols) Doing this effectively would require fairly tight coupling between the syntax of the language, ability to analyze dataflows relating to variables and function calls, cross-package dependency management, and perhaps other factors. Those properties can be achieved during design of a programming language, or they can iteratively be retrofitted into existing languages (with varying levels of difficulty). Bazel[1] attempts to achieve much of this, although to my understanding it offloads a lot of the work of determining (re)build requirements onto the developer - perhaps a necessary migration phase until more languages and environments provide formats that have self-evident out-of-date status and dependency graphs. We'll get there and the (sometimes uneasy) jokes will be about how often the software industry used to re-run huge continuous integration pipelines. [1] - https://en.wikipedia.org/wiki/Bazel_(software) https://en.wikipedia.org/wiki/Bazel_(software)
    - cntainer 4y ago
      Honestly diversity is a good thing. It's good that different people are trying to solve problems at the heart of software in different ways as it creates new perspectives and possibilities. Now, after reading these replies I'm really curious about stuff like darklang and Bazel. What happens is that people don't understand that most of these higher level solutions are very leaky abstractions, they aren't the silver bullet marketed in medium articles. Yes, they can save you a lot of time and headaches in specific scenarios but when you encounter one of the leaks it can take you weeks to get to the bottom of it. If the teams doesn't understand what problems the tool is solving and if they have that problem then they might be just cargo-culting. An example of this is kubernetes. I know teams that used kubernetes just because everybody else is using it, they didn't actually need it for their monolithic java spring app. They think they avoided accidental complexity by using kubernetes but in fact they just added accidental complexity to their project. And then they add a new tool that makes it easier to manage the complexity of kubernetes, and so on. Anyway, I'm probably just a rambling fool and I should appreciate that all this generating and shifting around of accidental complexity will actually mean future job safety for guys like us.
adalarmed 4y ago
> deferring all testing and linting to the CI process is an anti-pattern I'm confused as to why this is an anti-pattern? My understanding is that the CI pipeline should run unit tests and linting for every commit. But at the same time, developers should run their tests before pushing code.
- charcircuit 4y ago
  Linters and tests should pass before code is rebased onto master. YOLOing code onto master is an antipattern because if your code breaks the build then you are holding up everyone else from being able to make changes until yours gets reverted. If linters and tests are already passing there is a very good chance the rebase won't break anything.
  - viraptor 4y ago
    You can go a step further. Instead of merging anything, you can tell the ci to merge it. Then ci can make a "merge-test" branch, run all the tests on it, and if they pass, then ff-merge it to master for real. No need for "good chance" or rebasing just to keep up with master. It does take some extra work though, because GH and others don't really support this out of the box.
    - marcusarmstrong 4y ago
      Gitlab does support this with “Merged Result pipelines”[0]. We use them extensively alongside their merge train functionality to sequential is everything and it’s fantastic. [0]: https://docs.gitlab.com/ee/ci/pipelines/merged_results_pipelines.html https://docs.gitlab.com/ee/ci/pipelines/merged_results_pipel...
    - zelos 4y ago
      Isn’t this the way everyone works? Write code, run some tests/linting locally, then create a PR to master and the CI server runs all the tests and reports pass/fail. Changes without a pass can’t be merged to master.

benjiweber 4y ago

I think the point is about far shorter feedback loops than possible with today's pervasive tech. > something even more continuous than continuous integration. It might look like real-time integration of your changes, collaborative editing, or smarter build systems that produce near-live feedback e.g. https://medium.com/darklang/how-dark-deploys-code-in-50ms-771c6dd60671 https://medium.com/darklang/how-dark-deploys-code-in-50ms-77...

eterps 4y ago
Yes exactly, I was also thinking about how darklang does it. The feedback loop can be much shorter, and I'm sure it's not just limited to how darklang does it either.
cntainer 4y ago
Well, how short the feedback loop needs to be is something that can be controlled by the process, and the team sets the process. I can have the normal flow, which takes minutes and runs all the tests and other stuff I might have on my pipeline (security, performance, etc). I use this for typical day to day work on features. I don't care if the deploy takes 50ms or 5 minutes. I can have a fast-track for critical production patches. I skip all the main CI steps and just get my code quickly in production. If done right this takes seconds. I didn't know about dark but I've seen this type of promise too many times, so I'm pretty sure there are many tradeoffs hidden under the nice shiny exterior. I can't know until I try it, but that kind of complexity doesn't all just disappear, i always has a cost even if it's out of sight.
- jka 4y ago
  Here's the thing: we have hundreds of thousands of teams and organizations who currently each need to select and define those processes. Perhaps some of those teams and individuals are experts at determining the correct, minimal continuous integration suite to run on a per-commit basis to minimize time and energy expenditure without compromising correctness. But I can guarantee that not all (in fact, not many) are, and that they pay maintenance and mental overheads to adhere to those practices. It feels to me like there is a potentially massive opportunity to design an integrated language-and-infrastructure environment that only re-runs necessary checks based on each code diff. - "Altered a single-line docstring in a Python file? OK, the data-flow from that infers a rebuild of one of our documentation pages is required, let's do that... done in 2ms" - "Refactored the variable names and formatting within a C++ class without affecting any of the logic or ABI? OK, binary compatibility and test compatibility verified as unchanged, that's a no-op... done in 0ms" - "Renamed the API spec so that the 'Referer' header is renamed 'Referrer'? OK, that's going to invalidate approximately a million downstream server and client implementations, would you like to proceed?" (examples are arbitrary and should imply no specific limitations or characteristics of languages or protocols) Doing this effectively would require fairly tight coupling between the syntax of the language, ability to analyze dataflows relating to variables and function calls, cross-package dependency management, and perhaps other factors. Those properties can be achieved during design of a programming language, or they can iteratively be retrofitted into existing languages (with varying levels of difficulty). Bazel[1] attempts to achieve much of this, although to my understanding it offloads a lot of the work of determining (re)build requirements onto the developer - perhaps a necessary migration phase until more languages and environments provide formats that have self-evident out-of-date status and dependency graphs. We'll get there and the (sometimes uneasy) jokes will be about how often the software industry used to re-run huge continuous integration pipelines. [1] - https://en.wikipedia.org/wiki/Bazel_(software) https://en.wikipedia.org/wiki/Bazel_(software)
  - cntainer 4y ago
    Honestly diversity is a good thing. It's good that different people are trying to solve problems at the heart of software in different ways as it creates new perspectives and possibilities. Now, after reading these replies I'm really curious about stuff like darklang and Bazel. What happens is that people don't understand that most of these higher level solutions are very leaky abstractions, they aren't the silver bullet marketed in medium articles. Yes, they can save you a lot of time and headaches in specific scenarios but when you encounter one of the leaks it can take you weeks to get to the bottom of it. If the teams doesn't understand what problems the tool is solving and if they have that problem then they might be just cargo-culting. An example of this is kubernetes. I know teams that used kubernetes just because everybody else is using it, they didn't actually need it for their monolithic java spring app. They think they avoided accidental complexity by using kubernetes but in fact they just added accidental complexity to their project. And then they add a new tool that makes it easier to manage the complexity of kubernetes, and so on. Anyway, I'm probably just a rambling fool and I should appreciate that all this generating and shifting around of accidental complexity will actually mean future job safety for guys like us.

adalarmed 4y ago

> deferring all testing and linting to the CI process is an anti-pattern I'm confused as to why this is an anti-pattern? My understanding is that the CI pipeline should run unit tests and linting for every commit. But at the same time, developers should run their tests before pushing code.

charcircuit 4y ago
Linters and tests should pass before code is rebased onto master. YOLOing code onto master is an antipattern because if your code breaks the build then you are holding up everyone else from being able to make changes until yours gets reverted. If linters and tests are already passing there is a very good chance the rebase won't break anything.
- viraptor 4y ago
  You can go a step further. Instead of merging anything, you can tell the ci to merge it. Then ci can make a "merge-test" branch, run all the tests on it, and if they pass, then ff-merge it to master for real. No need for "good chance" or rebasing just to keep up with master. It does take some extra work though, because GH and others don't really support this out of the box.
  - marcusarmstrong 4y ago
    Gitlab does support this with “Merged Result pipelines”[0]. We use them extensively alongside their merge train functionality to sequential is everything and it’s fantastic. [0]: https://docs.gitlab.com/ee/ci/pipelines/merged_results_pipelines.html https://docs.gitlab.com/ee/ci/pipelines/merged_results_pipel...
  - zelos 4y ago
    Isn’t this the way everyone works? Write code, run some tests/linting locally, then create a PR to master and the CI server runs all the tests and reports pass/fail. Changes without a pass can’t be merged to master.