5 ms·

The situation you are describing is best handled by generative tests (i.e. Quickcheck). And this is basically the only time I would write an unit test - when I

by pka 11y ago

The situation you are describing is best handled by generative tests (i.e. Quickcheck). And this is basically the only time I would write an unit test - when I can not capture most invariants in the types. Everything else is basically double the work and a waste of time. Integration tests are always a good idea, but hard to get "conceptually" right. When you get to the level of testing multiple components through the whole stack most of the time IO is involved and this gets hairy fast, not to mention that one mostly tests the 3 codepaths they can come up with. Maybe a DSL/swappable interpreter approach is one way do that? Or make everything pure again with FRP?

duncanawoods 11y ago
I'm not a Haskell programmer but how would you deal with a case like e.g. using a set of complex regexes for parsing some unusual date-time formats. I would want to unit test the hell out of them covering all the variations and exceptions I could think of. I can't see how exhaustively testing a small unit like this with high-level tests would be practical in terms of setup effort and run-time.
- pka 11y ago
  In this particular case I would use some Quickcheck-like generative test library and give it a description of how to generate random dates (i.e. start by generating a day, then a month, then a combination of both etc) and then leave to it the generation of all test data. The amount of edge cases Quickcheck catches is beyond the most comprehensove unit test suite. On a higher level though I would use parser combinators. If you have a grammar specification you can then directly transform it into a parser and thus would only need to test that you transformed it correctly (i.e. prove the proof as opposed to prove the implementation if that makes sense). Basically - try to capture most of your invariants in the types and then you know your code is correct. The invariants may be wrong though, in which case integration testing may help. However, I still have no idea how to test programs with complex state and IO in a way that isn't basically "script the 5 behaviors I can come up with and pray for the best", in which case I mostly don't even bother (i.e. UI etc...) (I know this is a very short answer but I'm typing on a phone)
  - mattnewton 11y ago
    You can make a finite state machine to model your expectations on random input, but then your test code is almost as complex as the code :) I've found quickcheck super useful as a way to fuzz user input or find edge cases in unit tests I didn't see, but I still write all my integration tests by hand because they are so hard to formalize.

by pka 11y ago

duncanawoods 11y ago

I'm not a Haskell programmer but how would you deal with a case like e.g. using a set of complex regexes for parsing some unusual date-time formats. I would want to unit test the hell out of them covering all the variations and exceptions I could think of. I can't see how exhaustively testing a small unit like this with high-level tests would be practical in terms of setup effort and run-time.

pka 11y ago
In this particular case I would use some Quickcheck-like generative test library and give it a description of how to generate random dates (i.e. start by generating a day, then a month, then a combination of both etc) and then leave to it the generation of all test data. The amount of edge cases Quickcheck catches is beyond the most comprehensove unit test suite. On a higher level though I would use parser combinators. If you have a grammar specification you can then directly transform it into a parser and thus would only need to test that you transformed it correctly (i.e. prove the proof as opposed to prove the implementation if that makes sense). Basically - try to capture most of your invariants in the types and then you know your code is correct. The invariants may be wrong though, in which case integration testing may help. However, I still have no idea how to test programs with complex state and IO in a way that isn't basically "script the 5 behaviors I can come up with and pray for the best", in which case I mostly don't even bother (i.e. UI etc...) (I know this is a very short answer but I'm typing on a phone)
- mattnewton 11y ago
  You can make a finite state machine to model your expectations on random input, but then your test code is almost as complex as the code :) I've found quickcheck super useful as a way to fuzz user input or find edge cases in unit tests I didn't see, but I still write all my integration tests by hand because they are so hard to formalize.