11 ms·

What sort of tests don't you end up writing? I ask because I work in both dynamic and static languages, and find myself writing very similiar tests in each case

What sort of tests don't you end up writing? I ask because I work in both dynamic and static languages, and find myself writing very similiar tests in each case.

sanderjd 10y ago
Tests that verify the behavior of functions when they are passed data of a type that does not support the behavior that the function expects to be supported. I find this to be a very common case.
- dmichulke 10y ago
  I address this problem by checking the type at run-time via an assertion. Depending on where you are in the security vs efficiency spectrum, you may opt to ignore those assertions in production. Writing specific tests for that looks very inefficient to me but maybe I am missing something.
  - bbcbasic 10y ago
    I don't see how this reduces the need for a test that the wrong type is handled correctly.
    - dmichulke 10y ago
      The underlying idea is that it shouldn't handle the wrong type at all, it should fail. If some function A calls function B with a wrong type, the problem is with function A. So basically, there should be only type tests after IO functions (probably after parsing), the rest can be assertions.
      - mac01021 10y ago
        And you don't need tests to make sure the assertions are there?
        dmichulke 10y ago
        Interesting point :) At some point you have to assume something is there and correct (e.g., you don't write tests to check whether the tests are there, also you assume the testing tools work, ...). In static languages you also assume the programmer set the right type for the function argument, so you might as well assume to have an assertion in the critical code of the functional languages.
        bbcbasic 10y ago
        > At some point you have to assume something is there and correct If you excel at compositional design then this may work OK but a lot of code bases I have worked on just need a good ol' refactor to sort the mess out. And then the "there and correct" part could fail due to human error, or the new person not knowing the old assumptions etc. The asserts are a step in the right direction in documenting the assumptions though. Types go one level better by both documenting the interface, and enforcing that contract at compile time.
        winstonewert 10y ago
        Do you need a test to verify that your compiler correctly checks the type signature? (Actually, that could be occasionally useful in the face of implicit conversions)
        eropple 10y ago
        My compilers have test suites.
        winstonewert 10y ago
        And the compiler that generates the assert also has a test suite for that.
        bbcbasic 10y ago
        You can't get true 100% coverage* on digital hardware. (on quantum maybe!) so what to test or not is always a judgement call. Most of the time I'd assume the software stack is trustworthy but if you are doing something extreme you may want to test the compiler or even formally prove the compiler works as expected. * true as in all possible inputs and outputs rather than merely all code paths
  - fineline 10y ago
    Yes, you are missing a statically typed language. Tests or assertions? Neither. Security or efficiency? Both.
    - dmichulke 10y ago
      The grand-grandparent of your comment talked specifically about tests in static vs. dynamic languages. Your comment just solved the problem by ignoring the latter, not much of a contribution to say the least.
  - sanderjd 10y ago
    For sure, and if you're someone who loves runtime assertions for type mis-matches, then I think you'd really love static type assertions, which are basically the same thing, but checked earlier, making bugs they find cheaper. I agree that writing specific tests for that is very inefficient, and that few people (myself included) actually do it. I just also think the fact that people don't really write tests for a (anecdotally) common cause of bugs, and instead wait to see them in their production logs, is an anti-pattern, rather than a positive thing.
    - winstonewert 10y ago
      So, wait, what exactly is the common source of bugs? How does writing tests passing the incorrect types to function help catch them?
- winstonewert 10y ago
  I find this to be a rare case. Of course, YMMV. But I don't see or write tests in dynamic languages that verify behavior when I pass an inappropriate type. I trust that it'll produce an error when it tries to use operations that the type doesn't support.
  - sanderjd 10y ago
    Maybe I'm misunderstanding you, but it sounds like you mean that you trust it to produce that error when it's running (ie. in production). If that's the case, couldn't you do the same for all kinds of bugs and not bother writing tests at all? Put another way, what is different from this class of bug, which you don't think is worth testing, and other classes of bugs, which it sounds like you do?
    - winstonewert 10y ago
      Primarily specificity. I want the function to fail when it is passed the wrong arguments, but I don't really care about how exactly it fails. If the wrong types get passed to a function, I'm already screwed and there's no way to do the right thing at that point. I only care that I get enough information to debug what happened. In pretty much any other case, I want to function return some precise value or do some precise action. These are situations that the code will encounter during normal execution, and so I do care about what exactly it does.
lmm 10y ago
You might find https://spin.atomicobject.com/2014/12/09/typed-language-tdd-part1/ https://spin.atomicobject.com/2014/12/09/typed-language-tdd-... enlightening (note that in modern typed languages there's a lot less overhead to using type technique than there is in Java - but even in Java you can do some of this stuff)
- winstonewert 10y ago
  I'm afraid I found it confused. For the tests the check types of values, that's just a dumb test that you shouldn't write regardless of the language. Then for rest of the post, he talks about having nicely-defined tests which is orthogonal to whether or not your types are static.
  - lmm 10y ago
    > For the tests the check types of values, that's just a dumb test that you shouldn't write regardless of the language. The thing is, almost all the tests you'd write in a dynamic language end up being something you can encode into the types of values.
    - winstonewert 10y ago
      Almost all? That just sounds plain ridiculous. Can you actually give an example of a test that would be realistically written in a dynamic language that would be something encoded into the types of values?
      - lmm 10y ago
        You give me an example of a test that would realistically be written in a dynamic language then. I've been out of dynamic languages for so long that I've forgotten how to produce working code in them (seriously, I tried to write some Django because I used to be quite good at it and it seemed like a good fit for a specific problem, and I just wrote code that had stupid bugs everywhere).

What sort of tests don't you end up writing? I ask because I work in both dynamic and static languages, and find myself writing very similiar tests in each case.

sanderjd 10y ago

Tests that verify the behavior of functions when they are passed data of a type that does not support the behavior that the function expects to be supported. I find this to be a very common case.

dmichulke 10y ago
I address this problem by checking the type at run-time via an assertion. Depending on where you are in the security vs efficiency spectrum, you may opt to ignore those assertions in production. Writing specific tests for that looks very inefficient to me but maybe I am missing something.
- bbcbasic 10y ago
  I don't see how this reduces the need for a test that the wrong type is handled correctly.
  - dmichulke 10y ago
    The underlying idea is that it shouldn't handle the wrong type at all, it should fail. If some function A calls function B with a wrong type, the problem is with function A. So basically, there should be only type tests after IO functions (probably after parsing), the rest can be assertions.
    - mac01021 10y ago
      And you don't need tests to make sure the assertions are there?
      - dmichulke 10y ago
        Interesting point :) At some point you have to assume something is there and correct (e.g., you don't write tests to check whether the tests are there, also you assume the testing tools work, ...). In static languages you also assume the programmer set the right type for the function argument, so you might as well assume to have an assertion in the critical code of the functional languages.
        bbcbasic 10y ago
        > At some point you have to assume something is there and correct If you excel at compositional design then this may work OK but a lot of code bases I have worked on just need a good ol' refactor to sort the mess out. And then the "there and correct" part could fail due to human error, or the new person not knowing the old assumptions etc. The asserts are a step in the right direction in documenting the assumptions though. Types go one level better by both documenting the interface, and enforcing that contract at compile time.
      - winstonewert 10y ago
        Do you need a test to verify that your compiler correctly checks the type signature? (Actually, that could be occasionally useful in the face of implicit conversions)
        eropple 10y ago
        My compilers have test suites.
        winstonewert 10y ago
        And the compiler that generates the assert also has a test suite for that.
        bbcbasic 10y ago
        You can't get true 100% coverage* on digital hardware. (on quantum maybe!) so what to test or not is always a judgement call. Most of the time I'd assume the software stack is trustworthy but if you are doing something extreme you may want to test the compiler or even formally prove the compiler works as expected. * true as in all possible inputs and outputs rather than merely all code paths
- fineline 10y ago
  Yes, you are missing a statically typed language. Tests or assertions? Neither. Security or efficiency? Both.
  - dmichulke 10y ago
    The grand-grandparent of your comment talked specifically about tests in static vs. dynamic languages. Your comment just solved the problem by ignoring the latter, not much of a contribution to say the least.
- sanderjd 10y ago
  For sure, and if you're someone who loves runtime assertions for type mis-matches, then I think you'd really love static type assertions, which are basically the same thing, but checked earlier, making bugs they find cheaper. I agree that writing specific tests for that is very inefficient, and that few people (myself included) actually do it. I just also think the fact that people don't really write tests for a (anecdotally) common cause of bugs, and instead wait to see them in their production logs, is an anti-pattern, rather than a positive thing.
  - winstonewert 10y ago
    So, wait, what exactly is the common source of bugs? How does writing tests passing the incorrect types to function help catch them?
winstonewert 10y ago
I find this to be a rare case. Of course, YMMV. But I don't see or write tests in dynamic languages that verify behavior when I pass an inappropriate type. I trust that it'll produce an error when it tries to use operations that the type doesn't support.
- sanderjd 10y ago
  Maybe I'm misunderstanding you, but it sounds like you mean that you trust it to produce that error when it's running (ie. in production). If that's the case, couldn't you do the same for all kinds of bugs and not bother writing tests at all? Put another way, what is different from this class of bug, which you don't think is worth testing, and other classes of bugs, which it sounds like you do?
  - winstonewert 10y ago
    Primarily specificity. I want the function to fail when it is passed the wrong arguments, but I don't really care about how exactly it fails. If the wrong types get passed to a function, I'm already screwed and there's no way to do the right thing at that point. I only care that I get enough information to debug what happened. In pretty much any other case, I want to function return some precise value or do some precise action. These are situations that the code will encounter during normal execution, and so I do care about what exactly it does.

lmm 10y ago

You might find https://spin.atomicobject.com/2014/12/09/typed-language-tdd-part1/ https://spin.atomicobject.com/2014/12/09/typed-language-tdd-... enlightening (note that in modern typed languages there's a lot less overhead to using type technique than there is in Java - but even in Java you can do some of this stuff)

winstonewert 10y ago
I'm afraid I found it confused. For the tests the check types of values, that's just a dumb test that you shouldn't write regardless of the language. Then for rest of the post, he talks about having nicely-defined tests which is orthogonal to whether or not your types are static.
- lmm 10y ago
  > For the tests the check types of values, that's just a dumb test that you shouldn't write regardless of the language. The thing is, almost all the tests you'd write in a dynamic language end up being something you can encode into the types of values.
  - winstonewert 10y ago
    Almost all? That just sounds plain ridiculous. Can you actually give an example of a test that would be realistically written in a dynamic language that would be something encoded into the types of values?
    - lmm 10y ago
      You give me an example of a test that would realistically be written in a dynamic language then. I've been out of dynamic languages for so long that I've forgotten how to produce working code in them (seriously, I tried to write some Django because I used to be quite good at it and it seemed like a good fit for a specific problem, and I just wrote code that had stupid bugs everywhere).