6 ms·

Treat your CI system as a product for faster and better feedback

39 points by TalktoCrystal 3y ago

koromak 3y ago
But be careful, because if you're on a small team you'll inevitably become the CI guy. And your boss might not really appreciate that you're putting less product code out.
- koolba 3y ago
  More generally, if your boss does not appreciate or understand your contributions, then find a new boss. Unless of course you’re shirking and the lack of understanding works in your favor.
benjaminwootton 3y ago
I think the industry was moving in the right direction between approximately 2005-2015. There was a big emphasis on TDD, BDD, CI, Testing Pyramids, Mocking, Clean coding, dependency injection etc being led from the Java and Ruby communities. It felt more like engineering and that we were building on a solid base. I feel like we lost something as we moved to more cloud services, gluing frameworks together, and a lot more JavaScript in the wild.
- TalktoCrystal 3y ago
  The trend comes back around.
- doctor_eval 3y ago
  I’m sorry, but in my experience, dependency injection in Java was a total nightmare for reasoning and maintainability on anything but trivial projects. We just keep inventing special ways to make things more complex and magic, when what we really need is to accept that simplicity is a virtue.
  - ravenstine 3y ago
    Dependency injection isn't even needed if code is written functionally. After all, nothing needs to be "injected" if the function where the dependency being used isn't even called in the first place; you simply pass dependencies as arguments when needed. But communities like Java are so dedicated to OOP that highfalutin ideas like dependency injection are not only treated as necessities, but as good things. I don't see dependency injection as a good thing. It's complexity added to try and make up for other complexities. I've never seen a dependency injection system implemented in any language that wasn't unnecessarily difficult to understand or that didn't have gotchas. All of that wouldn't be needed if most things were written as functions (i.e. no ImageThumbnailHttpUploader class), or if class instances were always invoked directly without magic.
    - okeuro49 3y ago
      > or if class instances were always invoked directly without magic. Then you can't provide a test seam. Dependency injection + mocking is a very powerful concept.
      - BlargMcLarg 3y ago
        Sure you can. Class constructors are still a thing.
        bhdlr 3y ago
        > Sure you can. Class constructors are still a thing. Definitely. I do 99% of my dependency injection as constructor "injection". Injection frameworks and Whitebox testing just introduce layers of complexity that can be very difficult to debug
        bhdlr 3y ago
        > Sure you can. Class constructors are still a thing. Definitely. I do 99% of my dependency injection as constructor "injection". Injection frameworks and Whitebox testing just introduce layers of complexity that can be very difficult to debug
- pech0rin 3y ago
  I have started JavaScript/TypeScript development over the past year or so. Coming from the Ruby, Python, and Java community its really interesting to me to see how many of the same problems that say, Rails for example, solved about development are being re-hashed over and over again. I'm not a JS hater by any means, I find using something like Remix to be a really nice way to build web applications. Its mainly the whole ecosystem/community surrounding it that feels like its starting from scratch all the time that I find to be sort of infuriating. Sometimes I wanna shout THESE THINGS ARE ALREADY SOLVED. But maybe I'm becoming an old dude and just yelling at the servers.
  - HatchedLake721 3y ago
    "Rails on the front-end" has been solved by Rails members (particularly Yehuda Katz) – it’s Ember.js. Same philosophy – batteries included with a focus on developer productivity and don't reinvent the wheel. But it comes with the same catch as Rails. While Rails is not the most sexy and popular thing on the backend as Go/Node/Rust, same is Ember.js compared to React/Vue/Svelte/Next/etc. But after about 10 years of writing code, developers start to care less about sexiness and popularity and just want to provide value and build things with velocity and little friction, rather than arguing what folder structure to use or which router package to install.
    - lloeki 3y ago
      > Rails on the front-end" has been solved And that's when it's even needed at all, in many cases server side rendering is perfectly fine and serves the user better than a shoddy SPA justified by the only concern of SSR being "obsolete"
  - tacker2000 3y ago
    I feel the same as an old school PHP dipping into JS land occasionally. Lots of problems were solved, but of course somebody needed to create a fancy new “framework” and a new solution. Stuff like “static site generation” dont need over 300 new and different frameworks to solve it but i guess thats just engineering. We like to do it ourselves instead of “trusting” the old farts.
- ravenstine 3y ago
  People got a sour taste in their mouths because those things are associated with (though not necessarily dependent on) object-oriented programming, and OOP is not so cool anymore given how much unnecessary complexity it encourages. It's of course possible to do all those things you mentioned without OOP, but I think the overall culture of things like TDD, CI, dependency injection, etc., also happened to include OOP. After all, the languages you mentioned, Java and Ruby, are heavily object-oriented. The shift began to happen ~2011 when JavaScript started to become more capable, and web developers became more interested in progress within JavaScript than in actual software engineering. The JavaScript community also figured out, over the course of many years, that not every concept needs its own distinct class, hence why a lot of JavaScript frameworks and code in general today is more focused on getting reactivity and reusability down and much less about being like Rails. At the same time, JavaScript ruined things by not letting go of its culture of scrappy programming from back when things like jQuery were the only way to actually get anything done; JavaScript development became more about downloading a bunch of jQuery plugins and using them whether you understood them or not, and really not much is different in 2023 with how willing developers are to install an NPM package for the simplest of things. The latter can be partly blamed on the Ruby and Python communities, though, which encourage such behavior, but to a much lesser extent. Business objectives have been a big driving force behind the rise of software engineering and its gradual demise. After the 2008 recession, actually engineering software to maintain structural integrity while being flexible became of upmost importance, which partly explains why things like Rails became popular both inside and out of Silicon Valley. It seems like after ~2015 that these things became far less important to businesses in general; there was enough of a financial recovery that companies believed they could demand more of their own software while sweeping more technical debt under the rug by throwing more developers at their code. Business leaders, particularly those without much of a background in software, look at many aspects of software engineering and see them as antithetical to rolling out more features, and the incentives seem to inevitably lead to sacrificing maintainability. Though the individual software engineer may lament this, can we really say it hasn't worked out well enough? Have many businesses collapsed or declined because of bad code, if any? Nevertheless, it would be nice if the software community, or even just the web development community, could revisit actual engineering as opposed to more gee-whiz tools.
- golergka 3y ago
  Modern functional Typescript is the best mainstream language I've ever worked with. The only thing I'm ready to trade it for is Haskell.
- eddythompson80 3y ago
  It’s not JavaScript that killed the testing trend in my experience. It was direct and conscious push by management that simply doesn’t value any part of that coupled with very competitive hiring market around 2014-2021. Management, increasingly, only cares about time to market. At least within my area of the industry. Planning usually goes something like “how long you think this will take?” I say let’s say 3 months. “Why?”, I say well to do it right I’d like to change X first. Then implement a common Y layer. Verify and test them. Then do Z on top. “Let’s cut X. No need to invest in Y for now. We just want Z. Let’s budget testing/etc after. Can we do it in 1 month now?” Why waste time investing in CI, TDD, BDD, deterministic builds, etc when you know your management doesn’t value it, and you’ll be out of this role in 2 years? 2-3 years old code is considered legacy now.
  - Justsignedup 3y ago
    To be fair. Early startups don't know what will work. So investing into the future is pointless.
- thewix 3y ago
  I feel the same way. I started my career in the mid-00s in Enterprise software writing .NET. We used to have many conversations around modeling, patterns, testing, etc. I work at a Node shop now and none of that happens. I am trying to introduce strong typing with typescript and aspects of functional programming but it is very slow-going...
- l0b0 3y ago
  I mentioned Extreme Programming in a meeting recently, and nobody had even heard of it. :(
eliasmacpherson 3y ago
Yes, for example, atlassian have a bugtracker for bamboo, exemplary!: https://jira.atlassian.com/browse/BAM-21743?filter=98685 https://jira.atlassian.com/browse/BAM-21743?filter=98685 I'm sure the various product owners and stakeholders from product convene regularly to address long standing issues and whittle down the backlog, right?
akhayam 3y ago
Teams not respecting their CI systems will end up shipping slow, low quality code. That being said, the article referenced seemed to be talking about very basic and straightforward things. A far more nuanced and pragmatic perspective is required to get CI systems to work nicely with today’s CD (e.g. Argo, Flux, Spinnaker) and orchestration systems (e.g. k8s).
- ravenstine 3y ago
  Or individual developers will circumvent CI out of frustration with it, completely defeating its purpose and making it a waste of money.
  - akhayam 3y ago
    alas... this is very true. I still think a well maintained CI system which actively prunes flaky tests can be a great productivity booster for the team. Doing so does require serious and continued investment though, which most teams aren't willing to make.

39 points by TalktoCrystal 3y ago

But be careful, because if you're on a small team you'll inevitably become the CI guy. And your boss might not really appreciate that you're putting less product code out.

koolba 3y ago
More generally, if your boss does not appreciate or understand your contributions, then find a new boss. Unless of course you’re shirking and the lack of understanding works in your favor.

benjaminwootton 3y ago

I think the industry was moving in the right direction between approximately 2005-2015. There was a big emphasis on TDD, BDD, CI, Testing Pyramids, Mocking, Clean coding, dependency injection etc being led from the Java and Ruby communities. It felt more like engineering and that we were building on a solid base. I feel like we lost something as we moved to more cloud services, gluing frameworks together, and a lot more JavaScript in the wild.

TalktoCrystal 3y ago
The trend comes back around.
doctor_eval 3y ago
I’m sorry, but in my experience, dependency injection in Java was a total nightmare for reasoning and maintainability on anything but trivial projects. We just keep inventing special ways to make things more complex and magic, when what we really need is to accept that simplicity is a virtue.
- ravenstine 3y ago
  Dependency injection isn't even needed if code is written functionally. After all, nothing needs to be "injected" if the function where the dependency being used isn't even called in the first place; you simply pass dependencies as arguments when needed. But communities like Java are so dedicated to OOP that highfalutin ideas like dependency injection are not only treated as necessities, but as good things. I don't see dependency injection as a good thing. It's complexity added to try and make up for other complexities. I've never seen a dependency injection system implemented in any language that wasn't unnecessarily difficult to understand or that didn't have gotchas. All of that wouldn't be needed if most things were written as functions (i.e. no ImageThumbnailHttpUploader class), or if class instances were always invoked directly without magic.
  - okeuro49 3y ago
    > or if class instances were always invoked directly without magic. Then you can't provide a test seam. Dependency injection + mocking is a very powerful concept.
    - BlargMcLarg 3y ago
      Sure you can. Class constructors are still a thing.
      - bhdlr 3y ago
        > Sure you can. Class constructors are still a thing. Definitely. I do 99% of my dependency injection as constructor "injection". Injection frameworks and Whitebox testing just introduce layers of complexity that can be very difficult to debug
      - bhdlr 3y ago
        > Sure you can. Class constructors are still a thing. Definitely. I do 99% of my dependency injection as constructor "injection". Injection frameworks and Whitebox testing just introduce layers of complexity that can be very difficult to debug
pech0rin 3y ago
I have started JavaScript/TypeScript development over the past year or so. Coming from the Ruby, Python, and Java community its really interesting to me to see how many of the same problems that say, Rails for example, solved about development are being re-hashed over and over again. I'm not a JS hater by any means, I find using something like Remix to be a really nice way to build web applications. Its mainly the whole ecosystem/community surrounding it that feels like its starting from scratch all the time that I find to be sort of infuriating. Sometimes I wanna shout THESE THINGS ARE ALREADY SOLVED. But maybe I'm becoming an old dude and just yelling at the servers.
- HatchedLake721 3y ago
  "Rails on the front-end" has been solved by Rails members (particularly Yehuda Katz) – it’s Ember.js. Same philosophy – batteries included with a focus on developer productivity and don't reinvent the wheel. But it comes with the same catch as Rails. While Rails is not the most sexy and popular thing on the backend as Go/Node/Rust, same is Ember.js compared to React/Vue/Svelte/Next/etc. But after about 10 years of writing code, developers start to care less about sexiness and popularity and just want to provide value and build things with velocity and little friction, rather than arguing what folder structure to use or which router package to install.
  - lloeki 3y ago
    > Rails on the front-end" has been solved And that's when it's even needed at all, in many cases server side rendering is perfectly fine and serves the user better than a shoddy SPA justified by the only concern of SSR being "obsolete"
- tacker2000 3y ago
  I feel the same as an old school PHP dipping into JS land occasionally. Lots of problems were solved, but of course somebody needed to create a fancy new “framework” and a new solution. Stuff like “static site generation” dont need over 300 new and different frameworks to solve it but i guess thats just engineering. We like to do it ourselves instead of “trusting” the old farts.
ravenstine 3y ago
People got a sour taste in their mouths because those things are associated with (though not necessarily dependent on) object-oriented programming, and OOP is not so cool anymore given how much unnecessary complexity it encourages. It's of course possible to do all those things you mentioned without OOP, but I think the overall culture of things like TDD, CI, dependency injection, etc., also happened to include OOP. After all, the languages you mentioned, Java and Ruby, are heavily object-oriented. The shift began to happen ~2011 when JavaScript started to become more capable, and web developers became more interested in progress within JavaScript than in actual software engineering. The JavaScript community also figured out, over the course of many years, that not every concept needs its own distinct class, hence why a lot of JavaScript frameworks and code in general today is more focused on getting reactivity and reusability down and much less about being like Rails. At the same time, JavaScript ruined things by not letting go of its culture of scrappy programming from back when things like jQuery were the only way to actually get anything done; JavaScript development became more about downloading a bunch of jQuery plugins and using them whether you understood them or not, and really not much is different in 2023 with how willing developers are to install an NPM package for the simplest of things. The latter can be partly blamed on the Ruby and Python communities, though, which encourage such behavior, but to a much lesser extent. Business objectives have been a big driving force behind the rise of software engineering and its gradual demise. After the 2008 recession, actually engineering software to maintain structural integrity while being flexible became of upmost importance, which partly explains why things like Rails became popular both inside and out of Silicon Valley. It seems like after ~2015 that these things became far less important to businesses in general; there was enough of a financial recovery that companies believed they could demand more of their own software while sweeping more technical debt under the rug by throwing more developers at their code. Business leaders, particularly those without much of a background in software, look at many aspects of software engineering and see them as antithetical to rolling out more features, and the incentives seem to inevitably lead to sacrificing maintainability. Though the individual software engineer may lament this, can we really say it hasn't worked out well enough? Have many businesses collapsed or declined because of bad code, if any? Nevertheless, it would be nice if the software community, or even just the web development community, could revisit actual engineering as opposed to more gee-whiz tools.
golergka 3y ago
Modern functional Typescript is the best mainstream language I've ever worked with. The only thing I'm ready to trade it for is Haskell.
eddythompson80 3y ago
It’s not JavaScript that killed the testing trend in my experience. It was direct and conscious push by management that simply doesn’t value any part of that coupled with very competitive hiring market around 2014-2021. Management, increasingly, only cares about time to market. At least within my area of the industry. Planning usually goes something like “how long you think this will take?” I say let’s say 3 months. “Why?”, I say well to do it right I’d like to change X first. Then implement a common Y layer. Verify and test them. Then do Z on top. “Let’s cut X. No need to invest in Y for now. We just want Z. Let’s budget testing/etc after. Can we do it in 1 month now?” Why waste time investing in CI, TDD, BDD, deterministic builds, etc when you know your management doesn’t value it, and you’ll be out of this role in 2 years? 2-3 years old code is considered legacy now.
- Justsignedup 3y ago
  To be fair. Early startups don't know what will work. So investing into the future is pointless.
thewix 3y ago
I feel the same way. I started my career in the mid-00s in Enterprise software writing .NET. We used to have many conversations around modeling, patterns, testing, etc. I work at a Node shop now and none of that happens. I am trying to introduce strong typing with typescript and aspects of functional programming but it is very slow-going...
l0b0 3y ago
I mentioned Extreme Programming in a meeting recently, and nobody had even heard of it. :(

eliasmacpherson 3y ago

Yes, for example, atlassian have a bugtracker for bamboo, exemplary!: https://jira.atlassian.com/browse/BAM-21743?filter=98685 https://jira.atlassian.com/browse/BAM-21743?filter=98685 I'm sure the various product owners and stakeholders from product convene regularly to address long standing issues and whittle down the backlog, right?

akhayam 3y ago

Teams not respecting their CI systems will end up shipping slow, low quality code. That being said, the article referenced seemed to be talking about very basic and straightforward things. A far more nuanced and pragmatic perspective is required to get CI systems to work nicely with today’s CD (e.g. Argo, Flux, Spinnaker) and orchestration systems (e.g. k8s).

ravenstine 3y ago
Or individual developers will circumvent CI out of frustration with it, completely defeating its purpose and making it a waste of money.
- akhayam 3y ago
  alas... this is very true. I still think a well maintained CI system which actively prunes flaky tests can be a great productivity booster for the team. Doing so does require serious and continued investment though, which most teams aren't willing to make.