4 ms·

Unit tests are hardly the only kind of testing needed. First, they're completely decontextualized (hopefully), so they don't show the code working in any sort o

by beat 7y ago

Unit tests are hardly the only kind of testing needed. First, they're completely decontextualized (hopefully), so they don't show the code working in any sort of broader context. Second, they only show the code is consistent with itself, not consistent with requirements. More to the point, unit tests are tests by programmers for programmers, rather than by business for business. For any mildly complex modern app, you need integration tests to show context, and behavioral/functional tests (Cucumber) to show the code actually does what it's supposed to do according to the customer.

AmericanChopper 7y ago
> they don't show the code working in any sort of broader context If you’re testing a REST interface, then how much additional context you need? They’re supposed to be stateless, so they can be tested in isolation just fine, because that’s how they’re supposed to operate. Or does it stop being a unit test once your test includes DB interaction? For your typical web app, front end testing is the primary place you need anything more complicated, and even then you could debate how much of that should be manual.
- beat 7y ago
  Do you test timeouts? Connection drops? REST itself may be stateless, but it exists in a complex and failure-prone environment with numerous moving parts. What's the routing like between interfaces? Are there proxies? Do you control their configuration? Are your certs properly managed? "Unit tests should be sufficient" is a fancy way of saying "Throw it over the wall and pray it works".
  - AmericanChopper 7y ago
    Most of that sounds more like application performance monitoring than pipeline testing.
    - beat 7y ago
      Most of that sounds like things that can be tested for, rather than just tossing code over the wall and waiting to see if it catches fire.
      - AmericanChopper 7y ago
        Where’s this “throw it over the wall” stuff coming from? I write tests, if I push and my pipeline fails, I have to fix it. I’m rostered on call, if my app breaks, I have to get up in the middle of the night. For a simple web app (which is probably what most engineers on HN work on), unit testing is most of the testing you probably need to do (unless you think that a unit test that involves a DB interaction becomes an integration test). Front end code needs a little bit more than that, but automating front end tests is not particularly reliable to begin with, so you could really debate how much of that should be done by hand anyway. Some people swear by BDD frameworks, but that’s how you end up with a 4 hour test pipeline, without getting all that much additional confidence in your work.
        beat 7y ago
        The original discussion is about working in regulated environments like health care and banking, not youtubeforcats.com. BDD frameworks and four hour test pipelines can start sounding pretty good after things reach a certain level of complexity. I used to work on a federal banking system that moved a hundred billion dollars a DAY. Believe me, unit testing is not sufficient for that kind of world.

by beat 7y ago

AmericanChopper 7y ago

> they don't show the code working in any sort of broader context If you’re testing a REST interface, then how much additional context you need? They’re supposed to be stateless, so they can be tested in isolation just fine, because that’s how they’re supposed to operate. Or does it stop being a unit test once your test includes DB interaction? For your typical web app, front end testing is the primary place you need anything more complicated, and even then you could debate how much of that should be manual.

beat 7y ago
Do you test timeouts? Connection drops? REST itself may be stateless, but it exists in a complex and failure-prone environment with numerous moving parts. What's the routing like between interfaces? Are there proxies? Do you control their configuration? Are your certs properly managed? "Unit tests should be sufficient" is a fancy way of saying "Throw it over the wall and pray it works".
- AmericanChopper 7y ago
  Most of that sounds more like application performance monitoring than pipeline testing.
  - beat 7y ago
    Most of that sounds like things that can be tested for, rather than just tossing code over the wall and waiting to see if it catches fire.
    - AmericanChopper 7y ago
      Where’s this “throw it over the wall” stuff coming from? I write tests, if I push and my pipeline fails, I have to fix it. I’m rostered on call, if my app breaks, I have to get up in the middle of the night. For a simple web app (which is probably what most engineers on HN work on), unit testing is most of the testing you probably need to do (unless you think that a unit test that involves a DB interaction becomes an integration test). Front end code needs a little bit more than that, but automating front end tests is not particularly reliable to begin with, so you could really debate how much of that should be done by hand anyway. Some people swear by BDD frameworks, but that’s how you end up with a 4 hour test pipeline, without getting all that much additional confidence in your work.
      - beat 7y ago
        The original discussion is about working in regulated environments like health care and banking, not youtubeforcats.com. BDD frameworks and four hour test pipelines can start sounding pretty good after things reach a certain level of complexity. I used to work on a federal banking system that moved a hundred billion dollars a DAY. Believe me, unit testing is not sufficient for that kind of world.