5 ms·

Yeah, I firmly support testing, but if the code you are working on isn't explicitly written to make writing tests easier, than odds are (in my experience) that

by fhood 8y ago

Yeah, I firmly support testing, but if the code you are working on isn't explicitly written to make writing tests easier, than odds are (in my experience) that writing the tests will take longer than writing the code.

csours 8y ago
Yes I think this is the problem. The attitudes I mentioned come from people who tried testing and got fed up because of the many legitimate issues they had.
dylan604 8y ago
I have been known to write shitty code, but am always looking to make the next set of code less shitty or even redo something I'm working on if the time allows. I'm getting better, but have a long way to go. I have a feeling that my code would be bad to write tests against. What would entail making code easier to test against? Small bits of code meant to be included that does one specific thing? Wrapping that included coded into functions and/or objects and then creating a test suite to hit all methods etc?
- fhood 8y ago
  The issue I run into most often is that a system or set of functions I want to test relies on or resides in a complicated object that requires proper initialization. This means that to run tests I need to manually perform all of this initialization, including spoofing complicated internal data structures and whatnot. The fix? Better compartmentalization in a lot of cases. Or writing classes with a viable form of default initialization in mind.
- virgilp 8y ago
  I'm not a real proponent of TDD, but... try writing some tests first, when you develop a new feature. Or at least, write them sooner - before your implementation is "done"; write them when you think you figured out the design. See, tests are a new use of your API/interfaces - if you find it hard to test stuff that's important, maybe you didn't model the problem right? Rules of thumb: - Don't test implementation, test business workflows ("functional/component tests" are most important; unit tests are sometimes/often nice, but don't overdo it! If a simple refactoring breaks lots of tests, you're doing it wrong - testing implementation detail, not business logic); e2e tests are good and required, but are often slow and when they fail they don't necessarily always isolate the problem very well - Seek a functional coding style, once you get used you'll find out it is easier to test and easier to reason about (no state means you just test the logic, and it's easy to unit-test too) - Largely ignore code coverage (use it as a guideline to see whether there are important parts of your app that you ignored/ whether you forgot to add tests for some business workflows/ corner cases). - Avoid test hacks like calling private methods via reflection or whatnot. Remember, tests exercise your APIs - you either have the APIs wrong, or you're trying to test irrelevant implementation details. - Look for invariants, and test those. Things that should always be true. Often times, there are multiple acceptable results - avoid exact-match tests when that happens (e.g. if you make a speech-to-text system, don't test that audio clip X produces exact output Y; often times, a genuine improvement in the algorithm might break some of your tests). TBH I think correct testing is much harder than the industry gives it credit for. Maybe that's why it's so rarely encountered in practice :)
  - jdmichal 8y ago
    I'll take umbrage with your last point: "Look for invariants, and test those. Things that should always be true. Often times, there are multiple acceptable results - avoid exact-match tests when that happens (e.g. if you make a speech-to-text system, don't test that audio clip X produces exact output Y; often times, a genuine improvement in the algorithm might break some of your tests)." No, you should test for exactly the output you have coded to generate. Otherwise, you do not know when you have behavior regression. You would expect to have to update the text-to-speech tests when you modify the text-to-speech algorithm. But if you're modifying another algorithm, and you start seeing tests break in the text-to-speech algorithm, you're probably introducing a bug! A failed test means nothing other than the fact that you have changed behaviour -- and should therefore trigger on any behavioural change. It's your opportunity to vet the expectations of your changes against the actual behaviour of the changed system.
    - virgilp 8y ago
      I respectfully disagree. A test should fail, ideally, only when behavior change is undesirable (i.e. contracts are broken). Optimizations, new features etc. should not break existing tests, unless old functionality was affected. And then there's the whole thing about separating functional from performance concerns - even degraded performance shouldn't fail the functional tests. In fact, the example I gave was real-life - a friend from Google changed their speech recognition tests to avoid exact matches and it was a significant improvement in the life & productivity of the development team. [edit] There's also another damaging aspect of exact-match tests: they often test much more than what's intended. Take for instance a file conversion software (say from PDF to HTML). You add a feature to support SVG, and test it with various SVG shapes - it's easy & tempting to just run the software on an input PDF, check that the output HTML looks right (especially in the relevant SVG parts), and then add an exact-match test. Job done, yay! Except that, you do this a lot, and it will slow you down like hell. Because when it fails in the future, it's very hard to tell why (was the SVG conversion broken? or is it some unrelated thing, like a different but valid way to produce the output HTML?). Do this a lot and you won't be able to trust your tests anymore - any change and 400 of them fail, ain't nobody got time to check in depth what happened, "it's probably just a harmless change, let me take a cursory look and then I'll just update the reference to be the new output".
- jdmichal 8y ago
  There is a single cause of all unit-testing difficulty that I have ever seen: isolation. Functionality must be isolated to be testable. Examples: * You don't isolate side-effects, such as file creation. Now you must execute file creation every time you execute that functionality. Even though (I assume) you are not interested in testing the file system, you are now implicitly testing it. * You don't isolate external dependencies, such as requiring a database connection. Now you can't test without standing up a temporary database. * You don't isolate logic, such as doing several complicated validations in sequence within a single functionality. Now in order to test any of the validations, you must also arrange to pass all the previous validations.

by fhood 8y ago

csours 8y ago

Yes I think this is the problem. The attitudes I mentioned come from people who tried testing and got fed up because of the many legitimate issues they had.

dylan604 8y ago

I have been known to write shitty code, but am always looking to make the next set of code less shitty or even redo something I'm working on if the time allows. I'm getting better, but have a long way to go. I have a feeling that my code would be bad to write tests against. What would entail making code easier to test against? Small bits of code meant to be included that does one specific thing? Wrapping that included coded into functions and/or objects and then creating a test suite to hit all methods etc?

fhood 8y ago
The issue I run into most often is that a system or set of functions I want to test relies on or resides in a complicated object that requires proper initialization. This means that to run tests I need to manually perform all of this initialization, including spoofing complicated internal data structures and whatnot. The fix? Better compartmentalization in a lot of cases. Or writing classes with a viable form of default initialization in mind.
virgilp 8y ago
I'm not a real proponent of TDD, but... try writing some tests first, when you develop a new feature. Or at least, write them sooner - before your implementation is "done"; write them when you think you figured out the design. See, tests are a new use of your API/interfaces - if you find it hard to test stuff that's important, maybe you didn't model the problem right? Rules of thumb: - Don't test implementation, test business workflows ("functional/component tests" are most important; unit tests are sometimes/often nice, but don't overdo it! If a simple refactoring breaks lots of tests, you're doing it wrong - testing implementation detail, not business logic); e2e tests are good and required, but are often slow and when they fail they don't necessarily always isolate the problem very well - Seek a functional coding style, once you get used you'll find out it is easier to test and easier to reason about (no state means you just test the logic, and it's easy to unit-test too) - Largely ignore code coverage (use it as a guideline to see whether there are important parts of your app that you ignored/ whether you forgot to add tests for some business workflows/ corner cases). - Avoid test hacks like calling private methods via reflection or whatnot. Remember, tests exercise your APIs - you either have the APIs wrong, or you're trying to test irrelevant implementation details. - Look for invariants, and test those. Things that should always be true. Often times, there are multiple acceptable results - avoid exact-match tests when that happens (e.g. if you make a speech-to-text system, don't test that audio clip X produces exact output Y; often times, a genuine improvement in the algorithm might break some of your tests). TBH I think correct testing is much harder than the industry gives it credit for. Maybe that's why it's so rarely encountered in practice :)
- jdmichal 8y ago
  I'll take umbrage with your last point: "Look for invariants, and test those. Things that should always be true. Often times, there are multiple acceptable results - avoid exact-match tests when that happens (e.g. if you make a speech-to-text system, don't test that audio clip X produces exact output Y; often times, a genuine improvement in the algorithm might break some of your tests)." No, you should test for exactly the output you have coded to generate. Otherwise, you do not know when you have behavior regression. You would expect to have to update the text-to-speech tests when you modify the text-to-speech algorithm. But if you're modifying another algorithm, and you start seeing tests break in the text-to-speech algorithm, you're probably introducing a bug! A failed test means nothing other than the fact that you have changed behaviour -- and should therefore trigger on any behavioural change. It's your opportunity to vet the expectations of your changes against the actual behaviour of the changed system.
  - virgilp 8y ago
    I respectfully disagree. A test should fail, ideally, only when behavior change is undesirable (i.e. contracts are broken). Optimizations, new features etc. should not break existing tests, unless old functionality was affected. And then there's the whole thing about separating functional from performance concerns - even degraded performance shouldn't fail the functional tests. In fact, the example I gave was real-life - a friend from Google changed their speech recognition tests to avoid exact matches and it was a significant improvement in the life & productivity of the development team. [edit] There's also another damaging aspect of exact-match tests: they often test much more than what's intended. Take for instance a file conversion software (say from PDF to HTML). You add a feature to support SVG, and test it with various SVG shapes - it's easy & tempting to just run the software on an input PDF, check that the output HTML looks right (especially in the relevant SVG parts), and then add an exact-match test. Job done, yay! Except that, you do this a lot, and it will slow you down like hell. Because when it fails in the future, it's very hard to tell why (was the SVG conversion broken? or is it some unrelated thing, like a different but valid way to produce the output HTML?). Do this a lot and you won't be able to trust your tests anymore - any change and 400 of them fail, ain't nobody got time to check in depth what happened, "it's probably just a harmless change, let me take a cursory look and then I'll just update the reference to be the new output".
jdmichal 8y ago
There is a single cause of all unit-testing difficulty that I have ever seen: isolation. Functionality must be isolated to be testable. Examples: * You don't isolate side-effects, such as file creation. Now you must execute file creation every time you execute that functionality. Even though (I assume) you are not interested in testing the file system, you are now implicitly testing it. * You don't isolate external dependencies, such as requiring a database connection. Now you can't test without standing up a temporary database. * You don't isolate logic, such as doing several complicated validations in sequence within a single functionality. Now in order to test any of the validations, you must also arrange to pass all the previous validations.