5 ms·

> Often I have seen people put a lot of effort into testing things that really doesn't have much payback compared to what they could be testing instead. 90% of

• chriswarbo 5y ago
  > only blow up when you execute one particular branch at runtime On a related note: I'm a big fan of property-based testing, where each test is basically a function 'args => boolean', and we're asserting that it always returns true regardless of the given arguments (in reality there is usually more structure, for nicer error messages, skipping known edge cases, etc.). The test framework (e.g. Hypothesis, Quickcheck, Scalacheck, etc.) will run these functions on many different values, usually generated at random. This works really well for 'algorithmic' code, e.g. '(x: Foo) => parse(print(x)) == x', but it can sometimes be difficult to think of general statements we can make about more complicated systems. In that case, it's often useful to test "code FOO returns without error". This is essentially fuzzing, and can be useful to find such problematic branches (at least, the "obvious", low-hanging fruit cases).
  • erhk 5y ago
    Just use mocks for complicated systems
    • chriswarbo 5y ago
      No offense, but I think you've completely missed the point of what my comment was talking about. Mocks are a way to decouple components of a system during tests, e.g. to avoid having tests hammer a real database. I'm talking about the problem of figuring out what to test; in particular, which properties of a system are invariant. For example, lots of unit tests are based on coincidences, for example: { uid = createUser('Alice'); page = renderProfilePage(uid); assert(page.contains('Hello Alice')); } We can turn this into a property test by taking an arbitrary username as an argument: (username: String) => { uid = createUser(username); page = renderProfilePage(uid); assert(page.contains('Hello ' + username)); } However, this test will fail, since this is not an invariant of the system. In particular, it will fail when given an argument like "&<>", since the resulting page will not contain the text 'Hello &<>' (instead, it will contain "Hello &amp;<>"). All sorts of issues like this will crop up as we test more complicated code. For example, certain components might fail gracefully when given certain values (e.g. constructing a '404' return value); that might be perfectly correct behaviour, but it makes useful invariants harder to think of (since they must still hold, even in those cases!) Mocking is completely orthogonal to that. PS: I consider mocking to be a code smell. It can be very useful when trying to improve processes around a legacy system; but things which are designed to use mocking tend to be correlated with bad design.
      • mcharezinski 5y ago
        Common practise is to write own generators: (username: ArbUsernameValid) => ... ArbUsernameValid covers all relevant Strings that can be used here like take dictionary of names from 100 countries and feed you arb generator
      • erhk 5y ago
        I believe we are not on the same page for wnat we consider a complicated system to be. Do yo assume that $developer it the owner of all code in this mental exercise?
• rectang 5y ago
  > "unit testing is a magic bullet" cults The thing is, those unit tests achieve practical utility in those language environments, because without the protections of the type system those kinds of errors bring down systems all the time. I agree that more rigorous type systems can spare you from having to write a lot of boilerplate tests, and I prefer to work in more strictly typed language environments. Nevertheless, "unit" tests that prove the functionality of individual elements, even if you write fewer of them, still offer a huge ROI. They are damn close to a "magic bullet", and every project that neglects them pays the price.
• harlekein 5y ago
  Another common problem with unit testing is that the function being tested should actually be four different functions and the setup is so complex, it becomes more of an integration test on the wrong level. Then the developer will say "unit testing is a pain in the ass, I rather do integration tests". But the problem is that the code being tested is calling out its problems through these tests.
  • siva7 5y ago
    i hear this often but i don’t think it’s true. sometimes a good integration test will do you more favour than a dozen of unit tests. also unit tests can be a pain in the ass as a bad integration test
• dkersten 5y ago
  > 90% of the "unit tests" that I've observed in the wild are checking for things that a modern type system would easily prevent you from doing. I'm a heavy Clojure user, so I often hear how static types are unnecessary and not useful, dynamic types are much more convenient and aren't you using clojur.spec anyway to make sure your data is correct? But I'm a strong static type proponent and wish a statically typed Clojure existed. The reason is exactly as you say: with dynamic types, type errors are only found at runtime, which means the only way to find them ahead of time is to exercise the code thoroughly. So you rely on unit tests to find type errors. The problem with that is, testing is labour intensive and also non-exhaustive. You have to come up with all the ways the types might become mismatched and in my personal experience, its super easy that something slips through. These kinds of bugs seem to be the biggest cause of production errors in my code and they are the exact types of errors that a static type checker would have prevented. So I don't think unit tests area good substitute for a static type system. Many people argue that the overhead of static typing makes them too slow, is too much effort and makes it harder to write code. In my personal opinion, static types make me think about my data much more deeply and helps me design better software. Yes, its slower, but its slower because it forces me to think about the problem space more. The actual extra typing (har har) necessary to add type annotations is a very small overhead. If that's really what's making you slow, then consider learning to touchtype, switching to a better keyboard layout or better physical keyboard, an IDE with better autocompletion, just learn to use the slow down to think about your code more, or use a language with type inference. Actual writing code is a small part of my day anyway, so its not been an issue for me when I use a statically typed language. A problem I have with optional type hints is that they are optional: not all code (standard library, third party libraries) will have it, so you only get a small bit of the benefits.