5 ms·

There's an argument to be made that to the extent they break the rules in some manner, they're doing more harm than good. Should you be testing scenarios that a

There's an argument to be made that to the extent they break the rules in some manner, they're doing more harm than good. Should you be testing scenarios that are impossible to encounter with the code? Should frameworks be breaking encapsulation and contracts of code? Most of the time breaking the rules is a reflection of either a bad practice or lack of functionality in the language. I'm all for experimenting in either of those dimensions, but there's a lot of value to have a default behaviour of not doing that.

exabrial 2y ago
>There's an argument to be made that to the extent they break the rules in some manner, they're doing more harm than good. Should you be testing scenarios that are impossible to encounter with the code? Yes actually, especially when it comes to i/o. Let's say I want to simulate how many code handles a REST interaction with a third party service. I want to be able to mock the REST callout without performing any actual i/o. This is why these test frameworks are powerful and useful. Later if I want I can test to see if the actual callout works by hitting a local endpoint.
- cbsmith 2y ago
  "I want to be able to mock the REST callout without performing any actual i/o." Why? Because it's hard for you to mock out actual I/O, which could be solved by a tool that makes it easy to mock out actual I/O.
  - exabrial 2y ago
    We're not in the business of testing I/O. We outsource whether or not I/O is correct that to the people writing the JVM. Testing it as part of your test cases violates the "Test one thing and one thing exactly" principle. The goal is to reduce the surface area of the test as far as possible to distill what the author of the code wrote is correct.
    - cbsmith 2y ago
      That's an arbitrary division of "one thing" though. You've decided the "one thing" includes the HTTP request and response, but it doesn't involve the socket call for that request and response. I'm not saying the test has to involve a network, or a network card, or anything like that, but perhaps a more accurate reflection of the "unit" to be tested is something that reads and writes to a channel or even a socket (and again, not necessarily a TCP/IP socket). In reality, you're actually testing a bunch of method invocations that won't happen when the code is being used, and to do it, you're also breaking a lot of the contracts that one would otherwise expect to be maintained... maybe that's not the optimal way to do things.

exabrial 2y ago

>There's an argument to be made that to the extent they break the rules in some manner, they're doing more harm than good. Should you be testing scenarios that are impossible to encounter with the code? Yes actually, especially when it comes to i/o. Let's say I want to simulate how many code handles a REST interaction with a third party service. I want to be able to mock the REST callout without performing any actual i/o. This is why these test frameworks are powerful and useful. Later if I want I can test to see if the actual callout works by hitting a local endpoint.

cbsmith 2y ago
"I want to be able to mock the REST callout without performing any actual i/o." Why? Because it's hard for you to mock out actual I/O, which could be solved by a tool that makes it easy to mock out actual I/O.
- exabrial 2y ago
  We're not in the business of testing I/O. We outsource whether or not I/O is correct that to the people writing the JVM. Testing it as part of your test cases violates the "Test one thing and one thing exactly" principle. The goal is to reduce the surface area of the test as far as possible to distill what the author of the code wrote is correct.
  - cbsmith 2y ago
    That's an arbitrary division of "one thing" though. You've decided the "one thing" includes the HTTP request and response, but it doesn't involve the socket call for that request and response. I'm not saying the test has to involve a network, or a network card, or anything like that, but perhaps a more accurate reflection of the "unit" to be tested is something that reads and writes to a channel or even a socket (and again, not necessarily a TCP/IP socket). In reality, you're actually testing a bunch of method invocations that won't happen when the code is being used, and to do it, you're also breaking a lot of the contracts that one would otherwise expect to be maintained... maybe that's not the optimal way to do things.