3 ms·
I agree with you overall, in an ideal world we all have near unlimited compute power and even these "longer" e2e tests run in a few minutes, not hours. Unfortun
by Sodman 6y ago
I agree with you overall, in an ideal world we all have near unlimited compute power and even these "longer" e2e tests run in a few minutes, not hours. Unfortunately it's also not terribly uncommon to see these e2e suites take many hours, or sometimes even days to run.
I think the happy medium might be using a dependency-aware testing pipeline that only runs tests for areas affected by the current commit's changes. I've seen Bazel used in this kind of setup with some success, for example.
Ideally no bugs make it to production, but that's almost never the reality. Obviously it'll depend on your software's use-case and user demographics, but you could erode user-trust just as bad by having a trivial user-facing bug sit visible on your app/site for a few days or weeks because your CI/CD pipeline takes so long to run that it prevents you from pushing out timely releases.
- asdfasgasdgasdg 6y agoYeah, another aspect here is optimizing tests properly. Of course, this also takes engineering time, but if you get your performance culture in the right place, hopefully test run costs can be commensurate to the amount of value you're delivering. That will allow you to never have a situation where your tests take inordinate amounts of time because you cannot afford resources to run them. Everything is tradeoffs, of course. You have to decide what's right for your business. But I think it's rare that there is too much or too thorough of testing. I'm guessing most orgs err in the other direction.