3 ms·

Is CI even worth it anymore? To be honest, I prefer testing on my local machine. This seem unecessary.

filterfiber 3y ago
The CI shows the entire team that the tests are passing, and doesn't require other devs to pull it and run the tests themselves. It prevents devs from accidentally forgetting to run tests, or thinking "This is just a typo fix, it shouldnt break any tests". Tests might also run on multiple platforms/os's/architectures/versions, etc. The CD ensures that the code is only deployed after all tests passed, it was approved, merged, and ensures a deploy isn't skipped accidentally. Depending on what your doing it could take a while to fully build new binaries, build for multiple platforms/versions/architectures, etc.
madduci 3y ago
Yes it is very important, since not everyone has the discipline to run tests on their own machine, leading then to errors upstream.
tetromino_ 3y ago
Testing on your local machine won't catch portability issues (Linux vs macOS vs Windows, x64 vs Arm64). Unless you take great care in writing hermetic tests, testing on your local machine won't catch problems hidden by non-standard versions of libraries and tools in your local development environment (e.g. you installed modern versions of bash and python on your Mac, but your users are using what the OS comes with). And for large enough projects, running tests on a single machine is simply impractical: a full test suite run on a single machine would take hours; the only solution is to shard.
- kjkjadksj 3y ago
  You can set up arbitrary environments locally too.
  - misnome 3y ago
    Yeah! And set up scripts so that you don't need to remember how to do all the setup/building. And you could set up a trigger to build it when you make a commit so that you don't need to manually trigger every machine separately. And then you don't need CI!
  - yoav 3y ago
    You can forget to commit a file and it will work on your machine but break everything for your coworkers locally and in production. I see this class of issue often in early startups that don’t have robust CI implemented yet. Like literally once a month or more per company.
Jenk 3y ago
The whole point is to continuously integrate your work with that of others. If you are a solo operation then sure, no diff. On a team? Different story.
__alexs 3y ago
Then CI cluster has 100x the core count of my laptop, so yes.
jdblair 3y ago
Ah yes, the "works on my machine" testing technique
dataflow 3y ago
Maybe not for a small 1-person project. If you have a team project you definitely need automated tests to run regularly, and everyone needs to be able to see when/where they fail.
recursive 3y ago
Seems weird to put these in conflict. I do both. There's some overlap, but they seem to serve different purposes.
zimpenfish 3y ago
Some places it isn't possible to run the test suite on your local machine[1] (access to resources, sheer CPU power, etc.) [1] Unit tests, maybe, but not integration.
bluGill 3y ago
I'm waiting for a CI build as I write this - If I ran it locally it would take 3 hours, but the CI system can split across multiple AWS nodes and so be done in 25 minutes (the time of the longest build). I tried getting a more powerful machine, turns out my build becomes memory bandwidth bound at about 48 cores (building on a RAM disk makes the build take longer which is how I know it is memory bandwidth).
- ssdspoimdsjvv 3y ago
  In my case, it would take about 45 hours to run on a single machine. Our distributed testing infrastructure gets it done in about 2. But there's a limited amount of machines, so you usually have to add 2 more hours queuing to that.
mmmpetrichor 3y ago
Do you always remember to run every test? Does every other dev? Are there 100 other devs? If you check in bad code will it break something for 100 other devs? It all depends on the size of the org.
nextlevelwizard 3y ago
Depends what you are making, how many people depend on your code working and how many team members you have

The CI shows the entire team that the tests are passing, and doesn't require other devs to pull it and run the tests themselves. It prevents devs from accidentally forgetting to run tests, or thinking "This is just a typo fix, it shouldnt break any tests". Tests might also run on multiple platforms/os's/architectures/versions, etc. The CD ensures that the code is only deployed after all tests passed, it was approved, merged, and ensures a deploy isn't skipped accidentally. Depending on what your doing it could take a while to fully build new binaries, build for multiple platforms/versions/architectures, etc.

madduci 3y ago

Yes it is very important, since not everyone has the discipline to run tests on their own machine, leading then to errors upstream.

tetromino_ 3y ago

Testing on your local machine won't catch portability issues (Linux vs macOS vs Windows, x64 vs Arm64). Unless you take great care in writing hermetic tests, testing on your local machine won't catch problems hidden by non-standard versions of libraries and tools in your local development environment (e.g. you installed modern versions of bash and python on your Mac, but your users are using what the OS comes with). And for large enough projects, running tests on a single machine is simply impractical: a full test suite run on a single machine would take hours; the only solution is to shard.

kjkjadksj 3y ago
You can set up arbitrary environments locally too.
- misnome 3y ago
  Yeah! And set up scripts so that you don't need to remember how to do all the setup/building. And you could set up a trigger to build it when you make a commit so that you don't need to manually trigger every machine separately. And then you don't need CI!
- yoav 3y ago
  You can forget to commit a file and it will work on your machine but break everything for your coworkers locally and in production. I see this class of issue often in early startups that don’t have robust CI implemented yet. Like literally once a month or more per company.

Jenk 3y ago

The whole point is to continuously integrate your work with that of others. If you are a solo operation then sure, no diff. On a team? Different story.

__alexs 3y ago

Then CI cluster has 100x the core count of my laptop, so yes.

jdblair 3y ago

Ah yes, the "works on my machine" testing technique

dataflow 3y ago

Maybe not for a small 1-person project. If you have a team project you definitely need automated tests to run regularly, and everyone needs to be able to see when/where they fail.

recursive 3y ago

Seems weird to put these in conflict. I do both. There's some overlap, but they seem to serve different purposes.

zimpenfish 3y ago

Some places it isn't possible to run the test suite on your local machine[1] (access to resources, sheer CPU power, etc.) [1] Unit tests, maybe, but not integration.

bluGill 3y ago

I'm waiting for a CI build as I write this - If I ran it locally it would take 3 hours, but the CI system can split across multiple AWS nodes and so be done in 25 minutes (the time of the longest build). I tried getting a more powerful machine, turns out my build becomes memory bandwidth bound at about 48 cores (building on a RAM disk makes the build take longer which is how I know it is memory bandwidth).

ssdspoimdsjvv 3y ago
In my case, it would take about 45 hours to run on a single machine. Our distributed testing infrastructure gets it done in about 2. But there's a limited amount of machines, so you usually have to add 2 more hours queuing to that.

mmmpetrichor 3y ago

Do you always remember to run every test? Does every other dev? Are there 100 other devs? If you check in bad code will it break something for 100 other devs? It all depends on the size of the org.

nextlevelwizard 3y ago

Depends what you are making, how many people depend on your code working and how many team members you have