5 ms·

I never had any problems writing reliable end-to-end tests. They are super useful for catching serious subtle bugs before the system goes into production. Not h

I never had any problems writing reliable end-to-end tests. They are super useful for catching serious subtle bugs before the system goes into production. Not having solid end-to-end tests is a massive red flag for me.

amw-zero 5y ago
I flat out don’t believe you.
- mbrodersen 5y ago
  Well it is true. The fact that you don’t believe it tells me you have a lot to learn. Writing good end-to-end tests is a skill you need to learn. Don’t assume that software developers can do it without proper training/learning. It is hard to do well.
  - amw-zero 5y ago
    My disbelief of you is from my own experiences writing them, and talking to dozens of colleagues across different companies. Every single company has to heavily parallelize their e2e tests, and pays a huge CI bill on top of effort maintaining an overly complex CI config. Even after this, every company has to have a retry mechanism for their e2e tests because at least one fails at least every test run. It’s also the first thing I ask on interviews. I have many dat points on this one, across many teams and companies. It’s an abomination. Maybe you’re not talking about interactive web applications. Or maybe you’re not talking about the scale where you have thousands of e2e tests written over multi-year projects. If you’re talking about a toy project, sure you might not run into nondeterminism. These problems present themselves in aggregate.
    - mbrodersen 5y ago
      I am talking about very large scale systems with web + mobile + desktop clients in an Enterprise environments. Think airlines and airports (operations and pairing/rostering). However it sounds as if your experience has been pretty bad. So I understand where you are coming from. My experience is that yes it is hard to do right but also that it is worth doing. YMMV of course.
    - strokirk 5y ago
      I've never had any issues with the E2E tests in my company either, and I'm unsure where the flakyness would even come from. Using Cypress with great success.
firedark60 5y ago
+1 on this comment. We had issues with poorly written selenium tests and after rewriting new tests with cypress and better test practices, the e2e test are reliable enough to be used as canary testing in new environments without false positive flaking. It ultimately comes down to how much you're willing to invest in writing good tests. If you're suffering from seriously flakey e2e tests results, more often than not it's from either outdated tech, poor testing practices, and not enabling a selective retry on failures.
- amw-zero 5y ago
  You hit the nail on the head, using your own words. > selective retry on failures This is why your test suite passes, not because of an avoidance of outdated tech or poor practices. You rerun your flaky tests until they pass. That’s bad engineering, and the definition of non-determinism.

amw-zero 5y ago

I flat out don’t believe you.

mbrodersen 5y ago
Well it is true. The fact that you don’t believe it tells me you have a lot to learn. Writing good end-to-end tests is a skill you need to learn. Don’t assume that software developers can do it without proper training/learning. It is hard to do well.
- amw-zero 5y ago
  My disbelief of you is from my own experiences writing them, and talking to dozens of colleagues across different companies. Every single company has to heavily parallelize their e2e tests, and pays a huge CI bill on top of effort maintaining an overly complex CI config. Even after this, every company has to have a retry mechanism for their e2e tests because at least one fails at least every test run. It’s also the first thing I ask on interviews. I have many dat points on this one, across many teams and companies. It’s an abomination. Maybe you’re not talking about interactive web applications. Or maybe you’re not talking about the scale where you have thousands of e2e tests written over multi-year projects. If you’re talking about a toy project, sure you might not run into nondeterminism. These problems present themselves in aggregate.
  - mbrodersen 5y ago
    I am talking about very large scale systems with web + mobile + desktop clients in an Enterprise environments. Think airlines and airports (operations and pairing/rostering). However it sounds as if your experience has been pretty bad. So I understand where you are coming from. My experience is that yes it is hard to do right but also that it is worth doing. YMMV of course.
  - strokirk 5y ago
    I've never had any issues with the E2E tests in my company either, and I'm unsure where the flakyness would even come from. Using Cypress with great success.

firedark60 5y ago

+1 on this comment. We had issues with poorly written selenium tests and after rewriting new tests with cypress and better test practices, the e2e test are reliable enough to be used as canary testing in new environments without false positive flaking. It ultimately comes down to how much you're willing to invest in writing good tests. If you're suffering from seriously flakey e2e tests results, more often than not it's from either outdated tech, poor testing practices, and not enabling a selective retry on failures.

amw-zero 5y ago
You hit the nail on the head, using your own words. > selective retry on failures This is why your test suite passes, not because of an avoidance of outdated tech or poor practices. You rerun your flaky tests until they pass. That’s bad engineering, and the definition of non-determinism.