3 ms·

I'd also add: check out the branch locally. Your colleagues do some awful things to their local environments, and the central CI isn't perfect. Your IDE might h

I'd also add: check out the branch locally. Your colleagues do some awful things to their local environments, and the central CI isn't perfect. Your IDE might have more checks enabled than both, too I've worked in teams where I'm the only one who actually checks out the branch to poke around

siva7 1y ago
I strongly disagree on this one. The CI pipeline has to be the single source of truth. Your approach is a step back to the times of "but it works on my machine".
- hexbin010 1y ago
  Yes, ideally. Then again, I've caught plenty of bugs checking it out and running it locally. The CI pipeline may not wholly be in your team's control. I've worked at places where a 'devops' team had too much control, liked to tinker all the time and often broke stuff. Nothing we could do - devops were gods and didn't take input from anyone. Should I not do check it out locally and instead spend months and political capital arguing for more in the CI (which the PM would argue slows down velocity - and will win)? Pragmatism and workarounds are often employed in non-bigtech enterprise companies... edit: I also meant 'poke around' as in do a bit of informal QA. Again, your test and QA team might not be perfect...
- nabbed 1y ago
  There are times where it is reasonable, especially for bug fixes where there is a test to explicitly check that the bug is fixed. In that case, I might check out the code, revert the fix, and ensure that the test fails. There have been a few times I did that and the test still succeeded, so I can confirm that the test is not testing the right thing (in a very mature project, complexity or test settings can obscure what your test is actually doing, especially if it is an E2E test).
  - hexbin010 1y ago
    Yup, same here
  - esprehn 1y ago
    That can be an automated CI check that runs the new tests with the non-test files reverted though.
    - pavel_lishin 1y ago
      It can be! And maybe even should be. But I've never seen it. A great idea, though; when we finally get automated testing in CI, I'll certainly suggest it :)
    - Scarblac 1y ago
      That wouldn't necessarily compile though, eg if the fix involved changing a function signature or adding new functions.
  - globular-toast 1y ago
    I have done a similar thing. I teach juniors to follow a strict test-driven development style when handling bug fixes. Write the missing, and failing, test first, then make the tests pass. (I also recommend using this approach for everything). But you can't tell if they've actually done this. It makes me think that this should be automated. Seems fairly straightforward to do in test frameworks that keep the tests in separate files. Bit harder if they're in the same files, like Rust.
- Hackbraten 1y ago
  There can still be value in configuring your local IDE to emit more warnings than the centralized linter does, and looking at those warnings in your IDE. This can help you identify additional pieces of feedback that you might have missed by just reading the code in a web interface.
sevenseacat 1y ago
100%. I build web apps, and you can't automatically test every single possible interaction of a user on a page. Weird shit can very very easily sneak through without thorough manual testing.
dietr1ch 1y ago
In my mind you are wasting time and enabling your team to get away with not fix your CI. The worst part is the false sense of security that you get that will bite your team the second you miss reviewing a change or go on vacation
- 1718627440 1y ago
  If the code needs to work in 100 different environments it is likely that there are more bugs found, than if it is only tested in one standardised environment. The problem CI catches is, if that one environment isn't even standardised and reproducible.

siva7 1y ago

I strongly disagree on this one. The CI pipeline has to be the single source of truth. Your approach is a step back to the times of "but it works on my machine".

hexbin010 1y ago
Yes, ideally. Then again, I've caught plenty of bugs checking it out and running it locally. The CI pipeline may not wholly be in your team's control. I've worked at places where a 'devops' team had too much control, liked to tinker all the time and often broke stuff. Nothing we could do - devops were gods and didn't take input from anyone. Should I not do check it out locally and instead spend months and political capital arguing for more in the CI (which the PM would argue slows down velocity - and will win)? Pragmatism and workarounds are often employed in non-bigtech enterprise companies... edit: I also meant 'poke around' as in do a bit of informal QA. Again, your test and QA team might not be perfect...
nabbed 1y ago
There are times where it is reasonable, especially for bug fixes where there is a test to explicitly check that the bug is fixed. In that case, I might check out the code, revert the fix, and ensure that the test fails. There have been a few times I did that and the test still succeeded, so I can confirm that the test is not testing the right thing (in a very mature project, complexity or test settings can obscure what your test is actually doing, especially if it is an E2E test).
- hexbin010 1y ago
  Yup, same here
- esprehn 1y ago
  That can be an automated CI check that runs the new tests with the non-test files reverted though.
  - pavel_lishin 1y ago
    It can be! And maybe even should be. But I've never seen it. A great idea, though; when we finally get automated testing in CI, I'll certainly suggest it :)
  - Scarblac 1y ago
    That wouldn't necessarily compile though, eg if the fix involved changing a function signature or adding new functions.
- globular-toast 1y ago
  I have done a similar thing. I teach juniors to follow a strict test-driven development style when handling bug fixes. Write the missing, and failing, test first, then make the tests pass. (I also recommend using this approach for everything). But you can't tell if they've actually done this. It makes me think that this should be automated. Seems fairly straightforward to do in test frameworks that keep the tests in separate files. Bit harder if they're in the same files, like Rust.
Hackbraten 1y ago
There can still be value in configuring your local IDE to emit more warnings than the centralized linter does, and looking at those warnings in your IDE. This can help you identify additional pieces of feedback that you might have missed by just reading the code in a web interface.

sevenseacat 1y ago

100%. I build web apps, and you can't automatically test every single possible interaction of a user on a page. Weird shit can very very easily sneak through without thorough manual testing.

dietr1ch 1y ago

In my mind you are wasting time and enabling your team to get away with not fix your CI. The worst part is the false sense of security that you get that will bite your team the second you miss reviewing a change or go on vacation

1718627440 1y ago
If the code needs to work in 100 different environments it is likely that there are more bugs found, than if it is only tested in one standardised environment. The problem CI catches is, if that one environment isn't even standardised and reproducible.