5 ms·

There's a lot of room for nuance. If you "just read the fucking file" but the file isn't a "real" configuration file then isn't it just a "mock?" If you replace

There's a lot of room for nuance. If you "just read the fucking file" but the file isn't a "real" configuration file then isn't it just a "mock?" If you replace all network calls with an interceptor that forwards all calls and responses, and just check what's happening as a "listener," aren't you mocking out the network calls to a non-real implementation? At the end of the day, tests are necessarily a mock-up of what's real. You just happen to disagree with where some people put the abstraction layer. I also would like to make my tests more "real" but I have a lot of sympathy for folks that are trying to test something smaller without involving eg a file. After all, the whole point of "everything is a file" in Unix is that we shouldn't need to worry about this detail, it's an OS concern. If you write to a file that's not actually a file on disk but actually a device, that it should fundamentally be okay and work as expected.

miningape 2y ago
Yeah don't get me wrong, I'm not anti-mock - real code is messy, and the ideal of the same tests running everywhere will never work, so mocks are necessary. But I do think there's a lot more harm from over-mocking, than under-mocking. > file isn't a "real" configuration file then isn't it just a "mock?" I want to say "no" but I haven't thought about it enough yet. My reasoning is that the file itself contains information about the expected messages to/from systems, since it is the body of whatever the system should respond to. And while it is only 1 layer separated from just creating the same object in memory for your test this "feels" different because you can't just pull it out of your codebase into curl.
- IggleSniggle 2y ago
  Just to work this out together a little more in discussion form, since I appreciate your attitude: Consider these two scenarios: - read "test1-config.json" from disk, into whatever most easy JSON-adjacent format makes sense for your lang - just use the JSON-adjacent format directly Isn't the difference between these that one requires coupling input configuration of the environment to the tests (possibly inclusive of env vars and OS concerns around file I/o), making running the tests more confusing/complicated in aggregate, while the other requires coupling input configuration to the tests, making the unit under test clearer but potentially less reflective of the overall system? Effectively this is just an argument between integration tests and unit tests. Unit testers certainly have the rhetorical upper hand here, but I think the grug-brained developers among us feel that "the whole program should be a pure function." That can ultimately be reduced to a P-NP problem.
  - miningape 2y ago
    yeah - I don't think we should go so far as to write a config file for a test. But if we have something that is already readily convertible to/from json, it should be used. Not seeing it so much as a config for a test but as an argument we're storing in a separate file. For example if we had a dto that serialises to/from json we should be storing json not creating this dto manually - I would push it further and say any structure which is also easily/transformed from json, like extracting a certain property and using that in the test (although this is also context dependant, for example: if there are other tests using this same file). As a counter example I wouldn't advocate for using json config files to test something completely unrelated to an underlying json structure. > That can ultimately be reduced to a P-NP problem Yeah ideally the goal should be to write the simplest code possible, however we get there - shoehorning an approach is always going to add complexity. I think there's a lot of danger from taking rhetoric too far, sometimes we push an abstraction to its limits, when what's really required is a new perspective that works well at these limits. Effectively I think there's a range in which any argument is applicable, its a matter of assessing if the range is large enough, the rules simple enough, and it solves the actual problem at hand.
    - Izkata 2y ago
      > yeah - I don't think we should go so far as to write a config file for a test. But if we have something that is already readily convertible to/from json, it should be used. Not seeing it so much as a config for a test but as an argument we're storing in a separate file. This sounds like a great candidate for the facade pattern: https://en.m.wikipedia.org/wiki/Facade_pattern https://en.m.wikipedia.org/wiki/Facade_pattern Basically you hide dependencies behind a small interface, which lets you swap out implementations more easily. The facade is also part of your codebase rather than an external API, so it gives you something stable to mock. Rather than a building facade like the name is based on, I think of these as a stable foundation of things a module calls out to. Like your code is a box with an interface on one side (what tests and the rest of the codebase interact with) and the facade(s) are on the other side (dependencies/mocks of dependencies).
      - 2y ago

miningape 2y ago

Yeah don't get me wrong, I'm not anti-mock - real code is messy, and the ideal of the same tests running everywhere will never work, so mocks are necessary. But I do think there's a lot more harm from over-mocking, than under-mocking. > file isn't a "real" configuration file then isn't it just a "mock?" I want to say "no" but I haven't thought about it enough yet. My reasoning is that the file itself contains information about the expected messages to/from systems, since it is the body of whatever the system should respond to. And while it is only 1 layer separated from just creating the same object in memory for your test this "feels" different because you can't just pull it out of your codebase into curl.

IggleSniggle 2y ago
Just to work this out together a little more in discussion form, since I appreciate your attitude: Consider these two scenarios: - read "test1-config.json" from disk, into whatever most easy JSON-adjacent format makes sense for your lang - just use the JSON-adjacent format directly Isn't the difference between these that one requires coupling input configuration of the environment to the tests (possibly inclusive of env vars and OS concerns around file I/o), making running the tests more confusing/complicated in aggregate, while the other requires coupling input configuration to the tests, making the unit under test clearer but potentially less reflective of the overall system? Effectively this is just an argument between integration tests and unit tests. Unit testers certainly have the rhetorical upper hand here, but I think the grug-brained developers among us feel that "the whole program should be a pure function." That can ultimately be reduced to a P-NP problem.
- miningape 2y ago
  yeah - I don't think we should go so far as to write a config file for a test. But if we have something that is already readily convertible to/from json, it should be used. Not seeing it so much as a config for a test but as an argument we're storing in a separate file. For example if we had a dto that serialises to/from json we should be storing json not creating this dto manually - I would push it further and say any structure which is also easily/transformed from json, like extracting a certain property and using that in the test (although this is also context dependant, for example: if there are other tests using this same file). As a counter example I wouldn't advocate for using json config files to test something completely unrelated to an underlying json structure. > That can ultimately be reduced to a P-NP problem Yeah ideally the goal should be to write the simplest code possible, however we get there - shoehorning an approach is always going to add complexity. I think there's a lot of danger from taking rhetoric too far, sometimes we push an abstraction to its limits, when what's really required is a new perspective that works well at these limits. Effectively I think there's a range in which any argument is applicable, its a matter of assessing if the range is large enough, the rules simple enough, and it solves the actual problem at hand.
  - Izkata 2y ago
    > yeah - I don't think we should go so far as to write a config file for a test. But if we have something that is already readily convertible to/from json, it should be used. Not seeing it so much as a config for a test but as an argument we're storing in a separate file. This sounds like a great candidate for the facade pattern: https://en.m.wikipedia.org/wiki/Facade_pattern https://en.m.wikipedia.org/wiki/Facade_pattern Basically you hide dependencies behind a small interface, which lets you swap out implementations more easily. The facade is also part of your codebase rather than an external API, so it gives you something stable to mock. Rather than a building facade like the name is based on, I think of these as a stable foundation of things a module calls out to. Like your code is a box with an interface on one side (what tests and the rest of the codebase interact with) and the facade(s) are on the other side (dependencies/mocks of dependencies).
    - 2y ago