5 ms·

> In Execute Program, we always write tests around subsystems that either (1) are critical to the product or (2) contain a lot of details that can't be statical

> In Execute Program, we always write tests around subsystems that either (1) are critical to the product or (2) contain a lot of details that can't be statically checked. This is one of those things that sounds good when you're in college, and sounds, well, horrifying is too strong a word, but it will definitely set off the warning bells for anyone that's been programming for a decade or more. My team has a simple standard: every single user-facing requirement is verified by automated testing. We don't have 100% coverage, but it's damn close, somewhere like 98%. The remaining untested code is generally weird branch paths that are hard to trigger, and "shouldn't" be seen in production. But this goes beyond code coverage or TDD: firing up the debugger and executing a unit test is the simplest way to add a feature, period. I cannot fathom building and starting our application so that I can hand test every time I change a line of code. Unit and integration tests aren't punishment inflicted on inferior developers: they'd an extremely useful tool that makes your life easier and your team more productive.

noelwelsh 7y ago
What language do you use? My experience in Scala is that tests can be replaced with types in a number of cases (but not all!) In these cases it is often not worth implementing a test, IMO. Depending on the code it is possible to construct the types in such a way that an incorrect implementation is very hard. This requires a modern type system and the knowledge to use it effectively. Type systems like in Java and Go are not up the task for the most part.
- lern_too_spel 7y ago
  In Scala, you should test that the type system makes the guarantees you expect with compile time errors. Several frameworks provide tooling for this.
onei 7y ago
> Unit and integration tests aren't punishment inflicted on inferior developers: they'd an extremely useful tool that makes your life easier and your team more productive. I can't agree with this more and also find the reverse is true: not having tests will punish you repeatedly. If we measure inferiority by unproductivity, adding tests also enables you to become the opposite of inferior (ferior?)
- WesolyKubeczek 7y ago
  > become the opposite of inferior (ferior?) "superior"
51GWERY43 7y ago
> This is one of those things that sounds good when you're in college, and sounds, well, horrifying is too strong a word, but it will definitely set off the warning bells for anyone that's been programming for a decade or more. To someone that has programmed for a good 2 decades and more, the above sounds like what a college student or junior dev that just found out about TDD would say. Unit test and integration tests are (bad) substitutes for good and modular design, which should piggyback on a strong type system. I've seen it so often that whenever I read comments like the above I feel the need to create an HN account just to reply to them. Junior devs in charge of major design decisions create damage that goes way beyond their limited reach. Test code is untested code. Test code is brittle code - especially integration tests are. You need tests because you're using a dynamically typed language? Well, stop using that language for anything that is even remotely complex. Your future senior dev self will thank you.
- growse 7y ago
  A "good and modular design" with a "strong type system" won't help you when one of your dependencies has an upgrade with a subtle bug that changes the behaviour of your system.
- lucisferre 7y ago
  > test and integration tests are (bad) substitutes for good and modular design, which should piggyback on a strong type system This sounds completely hand wavy. If the design is in fact modular and built of autonomous components then it should be relatively easy to test and should be tested. Types are no substitute for testing and are completely orthogonal to testing. > Test code is brittle code If this is the case then the code being tested is more than likely brittle as well. Test code reflects the system under test more than anything else. > You need tests because you're using a dynamically typed language No you simply need tests in any language.
  - mixedCase 7y ago
    > Types are no substitute for testing. I don't agree with the person you're replying to that types make tests redundant, but to claim that types can't substitute a large amount of testing can only come from inexperience. Whatever property you can model successfully through a closed type, is a property you never have to test ever again anywhere it's being used. If you have 500 functions that take a NonEmptyString, those are 499 fewer property checks in tests (because you should have one test for the type constructor).
eyelidlessness 7y ago
> This is one of those things that sounds good when you're in college, and sounds, well, horrifying is too strong a word, but it will definitely set off the warning bells for anyone that's been programming for a decade or more. Speak for yourself :) I'm a huge TDD advocate, and I'm personally responsible for making testing a requirement of any dev work in my company. I am well over a decade in experience. Gary's quote sounds pretty close to my criteria for what to test (my criteria would replace "a lot of details that can't be statically checked" with "meaningful details that can't be statically checked"). The type system cannot replace all testing, but there are certainly places where it can make testing superfluous. Increasingly I strive to make more and more of a codebase like that. It improves confidence in the system, with a significantly faster red-green cycle, and isolates complexity to layers of an application where complexity is expected. > But this goes beyond code coverage or TDD: firing up the debugger and executing a unit test is the simplest way to add a feature, period. I cannot fathom building and starting our application so that I can hand test every time I change a line of code. In general, I agree with this, and in general this is my approach to programming. However, and this is a sincere question: do you work on web frontends? In my experience, some aspects of frontend work is absolutely easier to evaluate by simply running it. You may want to have automated tests for certain interactions, but even then as a regression test, as writing the test first is often harder than getting it working first and making sure the test exercises it after.
- thomascgalvin 7y ago
  > The type system cannot replace all testing, but there are certainly places where it can make testing superfluous. I agree. We configure analysis to ignore boilerplate code, for example, so that doesn't count toward out coverage threshold. And in this particular example, yeah, there is an entire class of tests that no longer need to be written; the "did he really pass me an integer, or is it JSON" style assertions. Typescript (and strong typing in general) is super helpful for eliminating that category of test. > However, and this is a sincere question: do you work on web frontends? When I'm forced to. I'd say my work in 80% backend, 20% frontend, though that ratio changes based on what stories we pick up for the week. But we use TDD for our frontend, too. I have selenium tests that bring up the backend, launch the web UI, and run through the required functionality. It's a very different mentality, and we don't check things like CSS layouts at this level, but the general "the user clicks this button and gets this result" are all verified automatically.
usefulcat 7y ago
Unit tests tend to be hyper-specific. Even after 20 years of writing unit tests, I find that getting the right balance between not enough testing and too much can still be quite challenging in some cases. For example, some things are a lot easier to test than others. Some things require a lot of setup that is tedious and error prone. In such cases, it's preferable to test at a lower level but it's not always possible to do that and still cover all relevant cases. Those are the cases where unit tests tend to be the most expensive. 100% test coverage may be desirable as a goal, but not at any cost, and tests absolutely do incur costs. So there is a trade-off, and it's worthy of consideration.

noelwelsh 7y ago

What language do you use? My experience in Scala is that tests can be replaced with types in a number of cases (but not all!) In these cases it is often not worth implementing a test, IMO. Depending on the code it is possible to construct the types in such a way that an incorrect implementation is very hard. This requires a modern type system and the knowledge to use it effectively. Type systems like in Java and Go are not up the task for the most part.

lern_too_spel 7y ago
In Scala, you should test that the type system makes the guarantees you expect with compile time errors. Several frameworks provide tooling for this.

onei 7y ago

> Unit and integration tests aren't punishment inflicted on inferior developers: they'd an extremely useful tool that makes your life easier and your team more productive. I can't agree with this more and also find the reverse is true: not having tests will punish you repeatedly. If we measure inferiority by unproductivity, adding tests also enables you to become the opposite of inferior (ferior?)

WesolyKubeczek 7y ago
> become the opposite of inferior (ferior?) "superior"

51GWERY43 7y ago

> This is one of those things that sounds good when you're in college, and sounds, well, horrifying is too strong a word, but it will definitely set off the warning bells for anyone that's been programming for a decade or more. To someone that has programmed for a good 2 decades and more, the above sounds like what a college student or junior dev that just found out about TDD would say. Unit test and integration tests are (bad) substitutes for good and modular design, which should piggyback on a strong type system. I've seen it so often that whenever I read comments like the above I feel the need to create an HN account just to reply to them. Junior devs in charge of major design decisions create damage that goes way beyond their limited reach. Test code is untested code. Test code is brittle code - especially integration tests are. You need tests because you're using a dynamically typed language? Well, stop using that language for anything that is even remotely complex. Your future senior dev self will thank you.

growse 7y ago
A "good and modular design" with a "strong type system" won't help you when one of your dependencies has an upgrade with a subtle bug that changes the behaviour of your system.
lucisferre 7y ago
> test and integration tests are (bad) substitutes for good and modular design, which should piggyback on a strong type system This sounds completely hand wavy. If the design is in fact modular and built of autonomous components then it should be relatively easy to test and should be tested. Types are no substitute for testing and are completely orthogonal to testing. > Test code is brittle code If this is the case then the code being tested is more than likely brittle as well. Test code reflects the system under test more than anything else. > You need tests because you're using a dynamically typed language No you simply need tests in any language.
- mixedCase 7y ago
  > Types are no substitute for testing. I don't agree with the person you're replying to that types make tests redundant, but to claim that types can't substitute a large amount of testing can only come from inexperience. Whatever property you can model successfully through a closed type, is a property you never have to test ever again anywhere it's being used. If you have 500 functions that take a NonEmptyString, those are 499 fewer property checks in tests (because you should have one test for the type constructor).

eyelidlessness 7y ago

> This is one of those things that sounds good when you're in college, and sounds, well, horrifying is too strong a word, but it will definitely set off the warning bells for anyone that's been programming for a decade or more. Speak for yourself :) I'm a huge TDD advocate, and I'm personally responsible for making testing a requirement of any dev work in my company. I am well over a decade in experience. Gary's quote sounds pretty close to my criteria for what to test (my criteria would replace "a lot of details that can't be statically checked" with "meaningful details that can't be statically checked"). The type system cannot replace all testing, but there are certainly places where it can make testing superfluous. Increasingly I strive to make more and more of a codebase like that. It improves confidence in the system, with a significantly faster red-green cycle, and isolates complexity to layers of an application where complexity is expected. > But this goes beyond code coverage or TDD: firing up the debugger and executing a unit test is the simplest way to add a feature, period. I cannot fathom building and starting our application so that I can hand test every time I change a line of code. In general, I agree with this, and in general this is my approach to programming. However, and this is a sincere question: do you work on web frontends? In my experience, some aspects of frontend work is absolutely easier to evaluate by simply running it. You may want to have automated tests for certain interactions, but even then as a regression test, as writing the test first is often harder than getting it working first and making sure the test exercises it after.

thomascgalvin 7y ago
> The type system cannot replace all testing, but there are certainly places where it can make testing superfluous. I agree. We configure analysis to ignore boilerplate code, for example, so that doesn't count toward out coverage threshold. And in this particular example, yeah, there is an entire class of tests that no longer need to be written; the "did he really pass me an integer, or is it JSON" style assertions. Typescript (and strong typing in general) is super helpful for eliminating that category of test. > However, and this is a sincere question: do you work on web frontends? When I'm forced to. I'd say my work in 80% backend, 20% frontend, though that ratio changes based on what stories we pick up for the week. But we use TDD for our frontend, too. I have selenium tests that bring up the backend, launch the web UI, and run through the required functionality. It's a very different mentality, and we don't check things like CSS layouts at this level, but the general "the user clicks this button and gets this result" are all verified automatically.

usefulcat 7y ago

Unit tests tend to be hyper-specific. Even after 20 years of writing unit tests, I find that getting the right balance between not enough testing and too much can still be quite challenging in some cases. For example, some things are a lot easier to test than others. Some things require a lot of setup that is tedious and error prone. In such cases, it's preferable to test at a lower level but it's not always possible to do that and still cover all relevant cases. Those are the cases where unit tests tend to be the most expensive. 100% test coverage may be desirable as a goal, but not at any cost, and tests absolutely do incur costs. So there is a trade-off, and it's worthy of consideration.