5 ms·
I always have to be careful responding to things like this because I get sucked into the trap of thinking “we” means myself, and it just needs to mean “most of
by hinkley 4y ago
I always have to be careful responding to things like this because I get sucked into the trap of thinking “we” means myself, and it just needs to mean “most of us” for whatever they’re talking about to be worrisome or even true.
I do worry that a lot of people are missing the first principles where CI is concerned, but a few of us still remember. Whether that’s enough is always a question.
> First, CI rarely knows what has changed, so everything needs to be rebuilt and retested
CI is the impartial judge in your system. That is the ultimate goal and much of the value. Don’t believe me when I say that your check in broke something. Believe the robot. It does not think, it does not interpret errors (which is why intermittent failures poison the well). It just knows that whatever you did was not good.
We make it stupid simple because the more ways you try to interpret things, the higher the likelihood the system will either miss an error or report imaginary ones. And in this case I am very sorry Mr Lasso, but it is in fact the hope that kills. Optimism makes us try things, things others would give up on. That’s what makes a piece of software good. Better. But too much and you start ignoring very real problems and make optimism into a liability. I’ve seen it over and over again, take a build with intermittent failures and people will check in broken code and assume it’s the intermittent problem and not their change.
Ideally, when you start breaking builds you’ve found your limits. But the build has to break when I break the code, and it needs to do it repeatably, because if I can’t reproduce the error I risk telling myself stories, and foisting off debugging to other people, which is a big no no.
- auxym 4y agoExactly this. CI gets rid of "works on my machine" syndrome. If it doesnt build on CI, you broke it, fix it before merging. Though, I'm still sour from my only experience in "tech" where "works on my machine" meant devs got to demand I fix the CI somehow.
- hinkley 4y agoIf self delusion is our biggest sin, learned helplessness is probably number two. A small but important part of my motivation to build better DevEx is that it puts a squeeze on the foot draggers. If you can’t blame the tools for being shit (and believe me, I know a lot of them are), then the only other explanations are that you are either too stupid to use them or just not a team player. An asshole. Developers will pick being called an asshole over being seen as stupid any day, but the former eventually gets management involved. It’s the closest I’ll probably ever get to solving a social problem with tech. Make it easy for people to participate, make them feel silly for not, then ramp up the pressure if that doesn’t work, and when it’s clear they only care about themselves you have actionable evidence to involve HR.
- lukeasrodgers 4y agoWhat if it was possible to have a local build system that would run tests etc and provide a cryptographically secure hash proving that the tests passed, the build was good, etc then a “centralized” CI system just verified that before permitting merge/deploy. There’s a lot of hand waving there but it doesn’t seem wildly implausible. Local use of containers or something like nixos could provide sufficient protection against the “works on my machine but not in production” problem.
- swagonomixxx 4y agoRunning tests locally is pretty much a no-no for many codebases because they are so large and tests take a long time if they don't run on a beefy test server or in parallel. Common examples of this are multi-million line C++ codebases (e.g proprietary game engines) and monorepos in any language. Running tests on my computer uses up valuable CPU cycles that I can use to work on something else while the CI servers are running my tests. However I can see this working for small to medium sized codebases that really don't need CI for testing.
- CuriouslyC 4y agoOr run a partial suite...
- swagonomixxx 4y agoPartial suite is certainly an option. You can build tools that only test code that has changed and whose changes can have cascading effects on other code (i.e dependent code). Do you know if such tools exist, or would I (as in the software engineer working on the codebase) have to build them?
- hinkley 4y agoPartial is the most valuable when you have red tests that are resisting simple fixes. I would definitely encourage running the entire tests every hour or two at the least. We have a project where the team split the tests into chunks and eventually I figured out the reason why is because they had coupling between tests, and running them all together ran into problems. I worry about other people opening that door, because it’s damned hard to close again.