4 ms·

> Inlining functions also has the benefit of not making it possible to call the function from other places. Congrats, you've got an untestable unit.

xboxnolifes 2y ago
The unit here is the email, not the email's link or subjects. Those are implementation details.
- mgsouth 2y ago
  What do you use unit tests for, other than verifying implementation details? Perhaps we have a difference in definition. To me, a unit test for a function such as "parse_news_email" would explore variations in parameters and states. Because of combinatorial explosion, that often means at least some white-box testing. I'm not going to generate random subjects and senders, and received-froms, I'm going to target based on internal details. Are we doing smart things with the message ID hostname? Then what happens if two messages come in with the same message ID but from different relays? The objective is that the unit test wrings out the implementation details, and the caller's unit test doesn't need to exhaustively test them. This white-box texting may require directly poking at or mocking internal functions or at least abusing how they're called. For example, parsing the news item might entail pulling up and modifying conversation thread cache entries or state. For some of the tests you may need hand-crafted cache state, it's not feasible to create unique states for each parameter combination you're testing, and testing a combination will pollute the state for the following combinations. Or maybe the function depends upon an external resource you can't beat to death with a million identical requests. So the least-bad, simplest solution may be to freeze or back out part of the normal state update in the unit test. Which would usually involve directly invoking the internal routines. Can this lead to fragile, false-positve to the point of useless tests? You betcha. That's where entertaining two contrary viewpoints is needed :) Use experience and good judgement about pros and cons in the particular situation.
  - randomdata 2y ago
    Unit tests are for documenting the API contract for the user. You are going to target based on what you are willing to forevermore commit to for those who will use what you have created. Indeed, what happens when two messages come in with the same message ID is something the user needs to be aware of and how it functions needs to remain stable no matter what you do behind the scenes in the future, so you would absolutely want to document that behaviour. How it is implemented is irrelevant, though. The only thing that matters is that, from the public API perspective, it is handled appropriately. There is a time and place for other types of tests, of course. You are right that unit tests are not the be all and end all. A good testing framework will allow you to mark for future developers which tests are "set in stone" and which are deemed throwaway.
    - mgsouth 2y ago
      We're in general agreement about the purpose of unit tests. I disagree on a couple of points. Tests do not document the API. No test is complete, and for that reason alone can't completely document anything. For example, a good API might specify that "the sender must be non-null, and must be valid per RFC blah." There's no way to test that inclusively, to check all possible inputs. You can't use the test cases to deduce "we must meet RFC blah." You might suspect it, but you'd be risking undefined behavior it you stray from input that doesn't exactly match the test cases. And before anyone objects "the API docs can be incomplete too," well, that true. But the point is that a written API has vastly more descriptive power than a set of test cases. (The same applies to "self-documenting code". Bah humbug.) There's also the objection "but you can't guarentee cases you don't test!" Also true. That's reality. _You can never test all your intended behavior._ You pick your test cases to do the best you can, and change your cases as problems pop up. The other thing I would shy away from is including throwaway tests in the framework. Throwaways are a thing, developers use them all the time, but don't make them unwanted stepchildren--poorly (incompletely?) designed, slapped together, limited scope, confusing for another developer (including time-traveling self) to wade through and decide whether this is a real failure or just bogus test. They're tech debt. Less frequently used tests are another matter. For example, release-engineering tests that only get run against release candidates. But these should be just as much real, set in stone, as any other deliverable. Which I guess is a third viewpoint nuance difference. I treat tests as being part of the package just as much as any other deliverable. They morph and shift as APIs change, or dependencies mutate, or bugs are found. They aren't something that can be put to the side and left to vegetate.
jeltz 2y ago
Which is usually a positive. Testing tiny subunits usually just makes refactoring and adding new features hard while not improving test quality.
- ninetyninenine 2y ago
  Not according to jon carmack. He stated he switched to pure functional programming in the intro which is basically stating all his logic is in the form of unit testable pure functions.
  - foldr 2y ago
    Nothing about pure functional programming requires unit testing all of your functions. You can decide to unit test larger or smaller units of code, just as you can in any other paradigm.
    - ninetyninenine 2y ago
      In pure functional programming a pure function is unit testable by definition of what a pure function is. I never said it requires functions to be tested. Just that it requires functions to be testable. In other paradigms do not do this. As soon as a module touches IO or state it becomes entangled with that and NOT unit testable. Is it still testable? Possibly. But not as a unit.
      - int_19h 2y ago
        How do you unit test a local function that is a closure in pure functional code?
        ninetyninenine 2y ago
        Of course you can't unit test things with restricted scope. f(x) = x + 2 + 4 How do you unit test x + 2 or (+ 4) even if the operation is pure? You can't. Because it's not callable. It's the same thing with the closure. The only things that are testable are things on unrestricted scope. AKA global scope. Think about what happens if you have a "closure" on global scope. If you really want to test it then your "unit tests" which typically live on global scope, need to be moved to local scope. That's just the rules of scope. There is one special case here. If the parent function returns the local function as a value. But even in this case the parent and local function have to be treated as a unit. The unit test will involve first calling the parent, then calling the local. The parent and child function form a "unit" thanks to shared state and the parent is essentially "moving" the local function into global scope. Generally best practice is to use combinators if you want to maximize the granularity in which you can modularize your logic. I would even argue that closures stradle the line between pure and impure, so I actually avoid closures whenever possible.
ninetyninenine 2y ago
This is a major insight. Defining a local function isn't a big deal you can always just copy and pasta it out to global scope. Any time you merge state with function you can no longer move the function. This is the same problem as OOP. Closures can't be modular the same way methods in objects can't be modular. The smallest unit of testable module is the combinator. John Carmack literally mentioned he does pure functional programming now which basically everyone in this entire thread is completely ignoring.
ahoka 2y ago
Congratulations, you are writing test for things that would not need test if weren't put behind a under-defined interface. Meanwhile sprint goals are not met and overall product quality is embarrassing, but you have 100% MC/DC coverage of your addNumbersOrThrowIfAbove(a, b, c).
gorgoiler 2y ago
Yup, and I should have called this out as a downside. Thank you for raising it. On visibility, one of the patterns I’ve always liked in Java is using package level visibility to limit functions to that code’s package and that packages tests, where they are in the same package (but possibly defined elsewhere.) (This doesn’t help though with the reduction in argument verbosity, of course.)
jayd16 2y ago
Ideally, you've just moved the unit boundary to where it logically should be instead of many small implementation details that should not be exposed.

xboxnolifes 2y ago

The unit here is the email, not the email's link or subjects. Those are implementation details.

mgsouth 2y ago
What do you use unit tests for, other than verifying implementation details? Perhaps we have a difference in definition. To me, a unit test for a function such as "parse_news_email" would explore variations in parameters and states. Because of combinatorial explosion, that often means at least some white-box testing. I'm not going to generate random subjects and senders, and received-froms, I'm going to target based on internal details. Are we doing smart things with the message ID hostname? Then what happens if two messages come in with the same message ID but from different relays? The objective is that the unit test wrings out the implementation details, and the caller's unit test doesn't need to exhaustively test them. This white-box texting may require directly poking at or mocking internal functions or at least abusing how they're called. For example, parsing the news item might entail pulling up and modifying conversation thread cache entries or state. For some of the tests you may need hand-crafted cache state, it's not feasible to create unique states for each parameter combination you're testing, and testing a combination will pollute the state for the following combinations. Or maybe the function depends upon an external resource you can't beat to death with a million identical requests. So the least-bad, simplest solution may be to freeze or back out part of the normal state update in the unit test. Which would usually involve directly invoking the internal routines. Can this lead to fragile, false-positve to the point of useless tests? You betcha. That's where entertaining two contrary viewpoints is needed :) Use experience and good judgement about pros and cons in the particular situation.
- randomdata 2y ago
  Unit tests are for documenting the API contract for the user. You are going to target based on what you are willing to forevermore commit to for those who will use what you have created. Indeed, what happens when two messages come in with the same message ID is something the user needs to be aware of and how it functions needs to remain stable no matter what you do behind the scenes in the future, so you would absolutely want to document that behaviour. How it is implemented is irrelevant, though. The only thing that matters is that, from the public API perspective, it is handled appropriately. There is a time and place for other types of tests, of course. You are right that unit tests are not the be all and end all. A good testing framework will allow you to mark for future developers which tests are "set in stone" and which are deemed throwaway.
  - mgsouth 2y ago
    We're in general agreement about the purpose of unit tests. I disagree on a couple of points. Tests do not document the API. No test is complete, and for that reason alone can't completely document anything. For example, a good API might specify that "the sender must be non-null, and must be valid per RFC blah." There's no way to test that inclusively, to check all possible inputs. You can't use the test cases to deduce "we must meet RFC blah." You might suspect it, but you'd be risking undefined behavior it you stray from input that doesn't exactly match the test cases. And before anyone objects "the API docs can be incomplete too," well, that true. But the point is that a written API has vastly more descriptive power than a set of test cases. (The same applies to "self-documenting code". Bah humbug.) There's also the objection "but you can't guarentee cases you don't test!" Also true. That's reality. _You can never test all your intended behavior._ You pick your test cases to do the best you can, and change your cases as problems pop up. The other thing I would shy away from is including throwaway tests in the framework. Throwaways are a thing, developers use them all the time, but don't make them unwanted stepchildren--poorly (incompletely?) designed, slapped together, limited scope, confusing for another developer (including time-traveling self) to wade through and decide whether this is a real failure or just bogus test. They're tech debt. Less frequently used tests are another matter. For example, release-engineering tests that only get run against release candidates. But these should be just as much real, set in stone, as any other deliverable. Which I guess is a third viewpoint nuance difference. I treat tests as being part of the package just as much as any other deliverable. They morph and shift as APIs change, or dependencies mutate, or bugs are found. They aren't something that can be put to the side and left to vegetate.

jeltz 2y ago

Which is usually a positive. Testing tiny subunits usually just makes refactoring and adding new features hard while not improving test quality.

ninetyninenine 2y ago
Not according to jon carmack. He stated he switched to pure functional programming in the intro which is basically stating all his logic is in the form of unit testable pure functions.
- foldr 2y ago
  Nothing about pure functional programming requires unit testing all of your functions. You can decide to unit test larger or smaller units of code, just as you can in any other paradigm.
  - ninetyninenine 2y ago
    In pure functional programming a pure function is unit testable by definition of what a pure function is. I never said it requires functions to be tested. Just that it requires functions to be testable. In other paradigms do not do this. As soon as a module touches IO or state it becomes entangled with that and NOT unit testable. Is it still testable? Possibly. But not as a unit.
    - int_19h 2y ago
      How do you unit test a local function that is a closure in pure functional code?
      - ninetyninenine 2y ago
        Of course you can't unit test things with restricted scope. f(x) = x + 2 + 4 How do you unit test x + 2 or (+ 4) even if the operation is pure? You can't. Because it's not callable. It's the same thing with the closure. The only things that are testable are things on unrestricted scope. AKA global scope. Think about what happens if you have a "closure" on global scope. If you really want to test it then your "unit tests" which typically live on global scope, need to be moved to local scope. That's just the rules of scope. There is one special case here. If the parent function returns the local function as a value. But even in this case the parent and local function have to be treated as a unit. The unit test will involve first calling the parent, then calling the local. The parent and child function form a "unit" thanks to shared state and the parent is essentially "moving" the local function into global scope. Generally best practice is to use combinators if you want to maximize the granularity in which you can modularize your logic. I would even argue that closures stradle the line between pure and impure, so I actually avoid closures whenever possible.

ninetyninenine 2y ago

This is a major insight. Defining a local function isn't a big deal you can always just copy and pasta it out to global scope. Any time you merge state with function you can no longer move the function. This is the same problem as OOP. Closures can't be modular the same way methods in objects can't be modular. The smallest unit of testable module is the combinator. John Carmack literally mentioned he does pure functional programming now which basically everyone in this entire thread is completely ignoring.

ahoka 2y ago

Congratulations, you are writing test for things that would not need test if weren't put behind a under-defined interface. Meanwhile sprint goals are not met and overall product quality is embarrassing, but you have 100% MC/DC coverage of your addNumbersOrThrowIfAbove(a, b, c).

gorgoiler 2y ago

Yup, and I should have called this out as a downside. Thank you for raising it. On visibility, one of the patterns I’ve always liked in Java is using package level visibility to limit functions to that code’s package and that packages tests, where they are in the same package (but possibly defined elsewhere.) (This doesn’t help though with the reduction in argument verbosity, of course.)

jayd16 2y ago

Ideally, you've just moved the unit boundary to where it logically should be instead of many small implementation details that should not be exposed.