4 ms·

> Well, the unit tests can find you a few logical errors, which is great. I write unit tests for that all the time. But all possible sources of error are discov

> Well, the unit tests can find you a few logical errors, which is great. I write unit tests for that all the time. But all possible sources of error are discovered by end-to-end tests. Citation needed, sounds anecdotal. If this really is how your app behaves then you're going to run into problems eventually because end-to-end tests are slow as well as difficult to write+maintain, and you'll need hundreds (if not thousands) of them to cover all the important business logic in your app. This will slow down your developer productivity (PRs take long to merge because people wait for CI to finish) and ability to deploy quickly. If your code is well-structured, your important business logic should be easily testable with unit and/or integration tests, and you don't need an end-to-end test to discover that, for example, a calculation is wrong. The pattern of functional core/imperative shell embodies this really well, I think.

deleted 6y ago
[deleted]
musingsole 6y ago
> end-to-end tests are slow as well as difficult to write+maintain Citation needed, sounds anecdotal. End-to-end tests should run as fast as any given use case (for most software, that means relatively quick), plus some set-up time as testing overhead. But that testing overhead is often self-inflicted and can be managed. I have anecdotes to prove it. For the speed to write end-to-end tests, shouldn't they be the easiest to capture? I need the system inputs and I need to capture the system outputs. If you imagine usage-driven development, then the last step is to capture your initial state, use your new code, capture the final state and voila, here's your new end-to-end test. > your important business logic Consider the reality that no one cares about your backend. No one. Is it correct enough for the application is the only business question concerned with the backend (and most applications have a HIGH fault tolerance). What does everyone care about? Your frontend. The colors, dropdown box or search field. That column should be over here. Focusing your testing on the parts of your application people could shrug about is not sound reasoning.
- Tainnor 6y ago
  I'm pretty sure more people care about the VAT being calculated correctly than about whether it's displayed on the left or on the right. You're taking a reasonable point of view - code matters only inasmuch as it has an effect on users - and take it to an absurd extreme.
  - musingsole 6y ago
    Then you misunderstand. You are right that the VAT or similar financial matters require a very high degree of precision and accuracy. Physics simulations and similar computations even more so. These applications have a very LOW tolerance for error and -- rightfully so -- tend to have a lot of attention paid to their backends. I contend that most applications are not these. I'll put it forward as professional experience that even financial adjacent systems have a surprising degree of error tolerance. Our engineers/scientists are constantly striving to improve correctness because that's what engineers do but it's plainly not a priority for our bottom line. Countless errors have been caught in the last year that made individual results quite questionable. But on the whole, the system and its results are accurate enough that we have plenty of customers and business keeping us busy.
pydry 6y ago
>If your code is well-structure You've alluded to the real benefit of end to end tests here, IMHO. They make very few demands about how your code is structured. This is fantastic if you've just been handed a big ball of mud. There is a definite trade off between faster running tests and tests which don't need to be rewritten every time you change some code.
- contravariant 6y ago
  Is that a benefit of end-to-end tests? Seems more like encouraging more unit tests would discourage bad coding habits if anything.
  - pydry 6y ago
    I don't find it does discourage bad coding habits. It just amplifies the problems they cause.
    - contravariant 6y ago
      Well I suppose any form of discouragement is pointless if it is not recognised and acted upon.
  - nullsense 6y ago
    As always in these discussions I like to remind everyone that you can have good and bad implementations of everything, and just doing unit testing doesn't make your engineers suddenly understand how to write testable code. So, you can well end up in a situation with a reasonable amount of code coverage, but not quite as high as you might like and when someone invariably digs into why they find while parts of the system are testable, parts of the system aren't. Now you're left with a large test suite that provides marginal value because it doesn't give you confidence your code can be deployed if just because it's passing, but the shite engineering practices heavily test implementation details which make maintenance slow to a crawl. Good engineering is what creates good engineering. What you actually do, I suspect matters a lot less.

deleted 6y ago

[deleted]

musingsole 6y ago

> end-to-end tests are slow as well as difficult to write+maintain Citation needed, sounds anecdotal. End-to-end tests should run as fast as any given use case (for most software, that means relatively quick), plus some set-up time as testing overhead. But that testing overhead is often self-inflicted and can be managed. I have anecdotes to prove it. For the speed to write end-to-end tests, shouldn't they be the easiest to capture? I need the system inputs and I need to capture the system outputs. If you imagine usage-driven development, then the last step is to capture your initial state, use your new code, capture the final state and voila, here's your new end-to-end test. > your important business logic Consider the reality that no one cares about your backend. No one. Is it correct enough for the application is the only business question concerned with the backend (and most applications have a HIGH fault tolerance). What does everyone care about? Your frontend. The colors, dropdown box or search field. That column should be over here. Focusing your testing on the parts of your application people could shrug about is not sound reasoning.

Tainnor 6y ago
I'm pretty sure more people care about the VAT being calculated correctly than about whether it's displayed on the left or on the right. You're taking a reasonable point of view - code matters only inasmuch as it has an effect on users - and take it to an absurd extreme.
- musingsole 6y ago
  Then you misunderstand. You are right that the VAT or similar financial matters require a very high degree of precision and accuracy. Physics simulations and similar computations even more so. These applications have a very LOW tolerance for error and -- rightfully so -- tend to have a lot of attention paid to their backends. I contend that most applications are not these. I'll put it forward as professional experience that even financial adjacent systems have a surprising degree of error tolerance. Our engineers/scientists are constantly striving to improve correctness because that's what engineers do but it's plainly not a priority for our bottom line. Countless errors have been caught in the last year that made individual results quite questionable. But on the whole, the system and its results are accurate enough that we have plenty of customers and business keeping us busy.

pydry 6y ago

>If your code is well-structure You've alluded to the real benefit of end to end tests here, IMHO. They make very few demands about how your code is structured. This is fantastic if you've just been handed a big ball of mud. There is a definite trade off between faster running tests and tests which don't need to be rewritten every time you change some code.

contravariant 6y ago
Is that a benefit of end-to-end tests? Seems more like encouraging more unit tests would discourage bad coding habits if anything.
- pydry 6y ago
  I don't find it does discourage bad coding habits. It just amplifies the problems they cause.
  - contravariant 6y ago
    Well I suppose any form of discouragement is pointless if it is not recognised and acted upon.
- nullsense 6y ago
  As always in these discussions I like to remind everyone that you can have good and bad implementations of everything, and just doing unit testing doesn't make your engineers suddenly understand how to write testable code. So, you can well end up in a situation with a reasonable amount of code coverage, but not quite as high as you might like and when someone invariably digs into why they find while parts of the system are testable, parts of the system aren't. Now you're left with a large test suite that provides marginal value because it doesn't give you confidence your code can be deployed if just because it's passing, but the shite engineering practices heavily test implementation details which make maintenance slow to a crawl. Good engineering is what creates good engineering. What you actually do, I suspect matters a lot less.