10 ms·

My personal opinion has become, do I want to write leaky tests to validate typing (in addition to functional tests) or do I want to let my compiler do my type v

• goto11 4y ago
  You shouldn't write test to validate types in the fist place. Testing for the correct behavior should be enough. I like static typing, but his this is just a silly argument. If the program behaves correctly according to spec, how can the types be wrong?
  • Yoric 4y ago
    I may be misunderstanding what you write, but I've had many (many, many) programs that worked for the wrong reasons, which led them to break in very (very, very) ugly ways due to unrelated changes. The only ways I know of to get rid of (some of) these bugs are to: - use as much static typing as possible; - wherever static typing is not sufficient, sprinkle this with a paranoid dose of assertions. If I read what you write correctly, that's way beyond "testing for the correct behavior", no?
    • goto11 4y ago
      > which led them to break in very (very, very) ugly ways due to unrelated changes. Sounds more like a problem with tight coupling than a problem with types per se. A function should not change behavior due to an "unrelated change", regardless of static or dynamic typing.
      • Yoric 4y ago
        Let me give you a trivial example (with TypeScript-style syntax, but assuming no TypeScript-style type-checking): ``` function foo(config: {hasYakShaver: boolean}) { if (config.hasYakShaver) { // ... } else { // ... } } foo({hasChuckYeager: false}); ``` This will accidentally do what I want, but for bad reasons. And one day, quite possibly, a change that appears unrelated will break this. And that's me being nice. If I start using reflection to do complicated stuff, or if I'm using C++ and the reason for which my code works is memory corruption... well, when I need to figure out what suddenly broke the code, I'm going to be up for serious a moment of loneliness.
        • goto11 4y ago
          Sure, static typing will catch such issues. But there is still no guarantee that the foo function (or the function calling foo) works correctly just because it passes the type checker. If you have unit tests (or other types of automated testing) you have a much bigger chance of catching logical errors and it will also catch trivial error like misspelling an identifier.
          • Yoric 4y ago
            I was responding to > Testing for the correct behavior should be enough. Also, in my experience, automated tests tend to catch many bugs, static typing also catches many bugs and defensive programming catches many bugs, too. There is an intersection between all three, but it's by no mean 100%. So I use all three of these. And of course, let's not discount monitoring, fuzzing, etc.
  • mjr00 4y ago
    > You shouldn't write test to validate types in the fist place. Testing for the correct behavior should be enough. Yeah but with dynamic typing there's a whole bunch of behaviors you need to think about. You pass an int into a method that expects a string, what's the expected behavior? Or maybe something more nefarious, like a method that expects a Python dictionary but is passed in a pandas Dataframe, which has similar syntax with totally different meaning; what's the expected behavior? Much easier to just specify what types you're expecting to work with and let a compiler or type checker deal with it.
    • skydhash 4y ago
      The two examples are human errors easily avoidable by reading the documentation. And some mechanism that uses types like method overload should be transparent from a behavior perspective.
      • mjr00 4y ago
        > The two examples are human errors easily avoidable by reading the documentation. Okay, but that doesn't answer the question. We're in dynamic typing world; my method accepts any data type by design. What's the expected behavior when it doesn't receive the expected type? Do I add assertions and fail with an assertion error? Let the runtime fail for me? Silently convert the type? In any case, I need to add a test for that. If your answer is "humans shouldn't make a mistake when calling the method," that's arguing for static typing, not against, because that's exactly what static typing prevents: a human making a mistake. Turns out static typing makes for a really good baseline level of documentation, as well. > And some mechanism that uses types like method overload should be transparent from a behavior perspective. Your method might work today for both a dict and a dataframe because it's only using simple accessors. But what if someone wants to go back and change it and they use something that's only on dictionaries, like the * operator? Your code would fine, your unit test cases probably all use dictionaries so they're fine, and it's only when you deploy to production and find out that someone's code elsewhere was incorrectly passing in a Dataframe that just happened to work, no longer does. Static typing has a solution for behavior-based types too, they're called interfaces. Even better in languages like Go and Python (with Protocols) that use structural type checking for interfaces, rather than explicit interface implementations.
  • gruffle 4y ago
    You're not going to cover all of the correct behavior in any nontrivial software with unit tests unless you plan to spend 99% of your time writing unit tests. Static typing gives a good baseline of correctness.
    • ReflectedImage 4y ago
      Static typing on its own is very bad at finding errors. Static typing is something you use for performance reasons primarily. If you want to catch errors with static typing you need Haskell or Rust levels of support for it. C# / Java / C++ don't make the grade.
      • peterashford 4y ago
        I really disagree with this. I use static types as a quick correctness check. I've added types to large python projects to reduce the complexity of reasoning about the code base, and the typing does not make it run faster (actually, the opposite).
        • ReflectedImage 4y ago
          A correctness check so quick, it didn't check correctness at all. That is not how to decrease the complexity of reasoning about a Python program. What you should do instead is break the program up into smaller microservices.
      • gruffle 4y ago
        You have no idea what you're talking about.
        • ReflectedImage 4y ago
          The issue is I know exactly what I'm talking about and you don't. Errors that would be caught by static type checking are exceptionally rare when you do commerical development in dynamically typed languages. You right now are complaining about a problem that only exists with inside your own head.
  • facorreia 4y ago
    Not OP, but as I understand it, it is about adding tests to check what the behavior will be if the input parameters have values outside of the expected type. E.g. what will be the behavior of "add(a, b)" if a or b are a string, null, array, object, undefined, etc. instead of numbers.
    • goto11 4y ago
      But why would you do that? Sounds like a waste of time. If some other function uses `add()` but pass an invalid argument, then this bug would be discovered by testing the function which pass the invalid argument. Presumably that function would work incorrectly.
      • mey 4y ago
        Say you have a python function that expects a dictionary, but instead a string gets passed in. At some point in the function the python function may or may not explode at runtime depending on how it's written. If your functional tests doesn't cover this scenario and you haven't guarded for invalid data types in your python code, you may never see this issue until it hits production or a refactoring of the code base is done. Ok, now that entire scenario goes away with a language that has a compilation phase and has type enforcement. It's boiler plate inside the function for type validation your don't have to do inside the python function (or blame the caller) and a testing scenario(s) you don't have to write. That's the work I'm talking about.
        • ReflectedImage 4y ago
          In practice, the function will be called indirectly via an API with types and there will be at least 1 unit test. Therefore the scenario is ridiculously unlikely and lives in the realm of fantasy.
        • goto11 4y ago
          The bug is in the function which passes a string instead of a dictionary. Presumably this function will work incorrectly, right? So the unit tests of that function will find the error. Unit tests which verifies a function has the correct behavior gives you "type checking" for free. You just have to shift the mindset to testing behavior instead of testing types.
  • spmurrayzzz 4y ago
    I'm glad I finally see this being raised. I don't know if I can make the claim that if a program behaves to spec, the types must be right (that seems like an academic statement out of my own wheelhouse). But I've always scratched my head at the canard of "static types saves time from obviating the need to write a whole class of type validation tests", that just doesn't map to my own experience. I challenged my intuitions on this recently and checked out some of my existing unit tests suites to see what my type error coverage looked like. It was exceedingly rare that I was able to fine a behavior assertion which after introducing an intentional change to cause a type error, didn't blow up the test. The cases where I did find type errors that snuck through with false negatives, the test itself was actually written poorly and the same behavior assertion would fail in other ways unrelated to types. I'm sure there are situations (in my own stack and in other folks') where my above findings are contradicted but, given that static typing for something like JS is not in any way a zero-cost abstraction, I'm not sure it's really a net win.
  • twblalock 4y ago
    Think of static typing as an autogenerated set of unit tests that ensure that for any given function, all arguments of the wrong type will be rejected. For example, if a function accepts a string as an input, test it by passing in all other possible primitives and classes and ensure that the behavior is strongly defined and results in the expected exception class. Then realize that if you haven't written all of those tests by hand in a dynamic codebase, you don't actually have code coverage. Anyone who works on your codebase could pass you an argument of the wrong type, and it might be in a code path that is rarely invoked and you will find out it's broken in prod.
    • goto11 4y ago
      > Think of static typing as an autogenerated set of unit tests that ensure that for any given function, all arguments of the wrong type will be rejected. My point is that you wouldn't write such unit tests in the first place. Unit tests should test behavior, not types.
      • twblalock 4y ago
        How a function reacts to the wrong type being passed as an input is part of its behavior. Otherwise the behavior is undefined, and that is really bad.
        • ReflectedImage 4y ago
          That's just a less likely case of giving a function the wrong input values, which is also undefined. So no it's not much worse than usual.
          • twblalock 4y ago
            No, it's actually different, because in a statically typed language it would not even be possible to write code that would pass the wrong types of inputs, because it would not compile! Also, if there are input values of the right type that cannot be handled properly, that should also be tested! That is the kind of thing tests are for! Example: you write a function that divides two numbers. It's not possible to divide by zero. You should absolutely have a test that passes in a divisor of zero and asserts that some kind of custom DivideByZeroException is thrown. If you know that certain inputs are out of range, you should test them explicitly -- otherwise you're being sloppy. If there is a large number of inputs out of range, you should test all of them, and you can use a `for` loop if you have to. The difference is, if you have a valid input out of range, that is all you need to test -- the language itself takes care of all the "wrong type" tests for you. Otherwise you have to write them yourselves, and if you don't, you don't actually have good coverage. I understand that it is customary in dynamic languages to not write those tests, but that is merely another argument for the superiority of static typing.
  • int_19h 4y ago
    Program behaving to the spec is something you'd write a functional test for; but OP is most likely talking about unit tests. If you're testing e.g. individual functions in isolation, you don't know how they're actually used on the outside, so you have to test for invalid inputs that the language allows - just like in a functional test, you have to test for invalid inputs coming from the environment. (There is a separate argument over what proportion of functional vs unit testing is optimal.)
• yamtaddle 4y ago
  I find static types invaluable as a machine-verifiable (to at least some degree) communication tool. It's mostly stuff you ought to be documenting anyway, but in a format that's way more useful than some notes in a readme, or a comment, or whatever. I find it alarming that folks claim to save time by avoiding them.
  • ReflectedImage 4y ago
    Static typing create a lot of code bloat, helper types, classes, abstract interfaces and all the sort. The increase in the code size usually results in significantly more bugs in the code. Number of bugs is directly proportional to the number of lines. The better way is to use unit testing instead.
    • yamtaddle 4y ago
      Interfaces and helper types and abstract interfaces and that sort of thing don't actually run, though... it'd be a neat trick to get anything resembling the bug rate of code-proper out of assistive structures like that. Besides, not all languages feature that so heavily in idiomatic code—not everything's Java. Anyway, that "bloat" is documentation your computer can understand so it can tell you when it's wrong. The alternative is documentation your computer can't check. Or just skipping that and letting the next person eat the higher onboarding cost and defect rate, which I suspect is more often the case. [EDIT] And it's not unit testing instead. It's in addition to. One's not a replacement for the other.
      • ReflectedImage 4y ago
        Each line of that compiler only documentation increases the number of bugs in your code. Bugs are directly proportional to total lines of code if you weren't aware
• akavi 4y ago
  I'm curious if you've had any experience with Swift/Kotlin on one hand and Typescript on the other. To me, these lie on the efficient frontier of the "compiler validation" vs "less boilerplate" tradeoff. Java is decidedly nowhere near that frontier.
  • mey 4y ago
    I like Typescript, it makes Javascript enjoyable to me. I use PEP 484 heavily in Python which help as well. I have no issues with Kotlin on the typing side of the house, val/var works well to infer types. I have issues with Kotlin in other areas (Jetbrains only ecosystem, threading (comes back to JS), compilation speed, personal stubbornness). I don't have any experience with Swift. Edit: A note on val/var w/ Kotlin, Intellij/Android Studio makes it easier to determine what the type of the variable is without having to dig for anything, looking upstream.