5 ms·

Perhaps I'm a masochist but I don't find these types of tests valuable. Fuzzing, e2e, property testing, & strict type system seem more worthwhile in my opinion

Perhaps I'm a masochist but I don't find these types of tests valuable. Fuzzing, e2e, property testing, & strict type system seem more worthwhile in my opinion.

doorman2 4y ago
Agreed. When you mock, you're essentially writing your implementation twice. Once for the real implementation, and again via the mock interactions. Sometimes, this type of test is helpful in the way that you can catch typos by writing the same piece of prose twice. However, these tests quickly become burdensome to maintain, e.g. changing an implementation in a way that produces identical results requires updating tests even though the behavior is unchanged.
- postalrat 4y ago
  I hate mocks. But all testing is about writing at least your intention twice and hoping you got it right at least once.
pan69 4y ago
I didn't read the full article (yet), I only skimmed it so far. I think the author is trying to demonstrate one or more principles, he/she does that with simple examples so not to overload the reader. So, yes. Glancing over the examples, they might seem futile but I would assume that is a much larger application with a lot more complexity that the principles demonstrated become more useful. Another example of this is explaining Object Oriented Programming, which with simple examples seem bloated and can be achieved with a much simpler approach. However, in the context of a large, decoupled enterprise system its a totally different thing.
oxff 4y ago
Mocking almost certainly is always the wrong thing to do. The maintenance burden alone from making sure the mock accuracy wrt. real implementation is high enough to guarantee high enough assurance should disqualify using them. They also make the tests nearly unreadable and very hard to reason about IME.
- lamontcg 4y ago
  > To test a piece of code swiftly and have consistent reliable results all the other dependencies have to be replaced with mocks controlled by the software engineer writing the tests. For example, if you are testing a function that relies on a file in the file system or makes an HTTP call over the network these external resources must be replaced with mocks for the tests to be a unit test. Relying on any external resource can make the test results flaky and hence unreliable. For both of those examples you can: 1. Create a temp file in a temp dir and inject the tempfile (probably the name of the tempfile) into the object under test. So if your object edits and manipulates /etc/passwd you can create a real file with some passwd contents (or a built-in fixture checked into the tests) and then point the object at that instead of /etc/passwd. Then you don't have to mock the whole POSIX API. Since you create the file you can rely on it existing. 2. Write a minimal API which runs on localhost with minimal or no state, no threading, possibly no auth (although something elsewhere in your tests needs to test auth) or anything else, which your code can setup expectations and responses on then point the object at the "server". Since you create this service in your tests you can rely on it existing and it shouldn't be brittle. For other objects you can expand the System-Under-Test (SUT) so that you construct multiple objects and feed them into the test. This is particularly true when the object you are testing sends messages to other objects and then expects to get changed state back out of them. Often better to just construct a real object rather than a mock. In a perfect world you might refactor everything so perfect unit testing was possible, but in reality you just won't be able to do this. And in general, unit tests are often useless because of the mocking and because if the contract changes on one side and not the other then you wind up shipping broken code. They're fast, which is great, which means you can enumerate all of the edge cases, but some kind of functional/integration testing that uses larger bits of the system together is almost always better in terms of giving confidence that you're shipping code that works. And with an infinitely fast CI system I'd argue that you should never write unit tests and you should be spinning up real APIs in your test harness and hitting them with real client workflows. Instead I write a lot of fast unit tests, although I'm real quick to jettison the idea that a unit test only covers one single object under test, and if the result isn't "really" a unit test, I'll let the philosophers sort it out.
  - 4y ago
Gabriel_h 4y ago
You may find this interesting too https://meticulous.ai/blog/let-users-write-tests-for-you/ https://meticulous.ai/blog/let-users-write-tests-for-you/ r.e. e2e testing and automatically generating mocks for UI tests. Maintaining mocks in e2e tests often becomes infeasible with a meaningful amount of e2e tests.
paulddraper 4y ago
Accurate. Unit tests are valuable if you're writing something with a very well defined interface like a base64 encoder. But bugs usually happen in the "glue". Static checks and e2e pound for pound catch the most bugs with the least effort.

Perhaps I'm a masochist but I don't find these types of tests valuable. Fuzzing, e2e, property testing, & strict type system seem more worthwhile in my opinion.

doorman2 4y ago

Agreed. When you mock, you're essentially writing your implementation twice. Once for the real implementation, and again via the mock interactions. Sometimes, this type of test is helpful in the way that you can catch typos by writing the same piece of prose twice. However, these tests quickly become burdensome to maintain, e.g. changing an implementation in a way that produces identical results requires updating tests even though the behavior is unchanged.

postalrat 4y ago
I hate mocks. But all testing is about writing at least your intention twice and hoping you got it right at least once.

pan69 4y ago

I didn't read the full article (yet), I only skimmed it so far. I think the author is trying to demonstrate one or more principles, he/she does that with simple examples so not to overload the reader. So, yes. Glancing over the examples, they might seem futile but I would assume that is a much larger application with a lot more complexity that the principles demonstrated become more useful. Another example of this is explaining Object Oriented Programming, which with simple examples seem bloated and can be achieved with a much simpler approach. However, in the context of a large, decoupled enterprise system its a totally different thing.

oxff 4y ago

Mocking almost certainly is always the wrong thing to do. The maintenance burden alone from making sure the mock accuracy wrt. real implementation is high enough to guarantee high enough assurance should disqualify using them. They also make the tests nearly unreadable and very hard to reason about IME.

lamontcg 4y ago
> To test a piece of code swiftly and have consistent reliable results all the other dependencies have to be replaced with mocks controlled by the software engineer writing the tests. For example, if you are testing a function that relies on a file in the file system or makes an HTTP call over the network these external resources must be replaced with mocks for the tests to be a unit test. Relying on any external resource can make the test results flaky and hence unreliable. For both of those examples you can: 1. Create a temp file in a temp dir and inject the tempfile (probably the name of the tempfile) into the object under test. So if your object edits and manipulates /etc/passwd you can create a real file with some passwd contents (or a built-in fixture checked into the tests) and then point the object at that instead of /etc/passwd. Then you don't have to mock the whole POSIX API. Since you create the file you can rely on it existing. 2. Write a minimal API which runs on localhost with minimal or no state, no threading, possibly no auth (although something elsewhere in your tests needs to test auth) or anything else, which your code can setup expectations and responses on then point the object at the "server". Since you create this service in your tests you can rely on it existing and it shouldn't be brittle. For other objects you can expand the System-Under-Test (SUT) so that you construct multiple objects and feed them into the test. This is particularly true when the object you are testing sends messages to other objects and then expects to get changed state back out of them. Often better to just construct a real object rather than a mock. In a perfect world you might refactor everything so perfect unit testing was possible, but in reality you just won't be able to do this. And in general, unit tests are often useless because of the mocking and because if the contract changes on one side and not the other then you wind up shipping broken code. They're fast, which is great, which means you can enumerate all of the edge cases, but some kind of functional/integration testing that uses larger bits of the system together is almost always better in terms of giving confidence that you're shipping code that works. And with an infinitely fast CI system I'd argue that you should never write unit tests and you should be spinning up real APIs in your test harness and hitting them with real client workflows. Instead I write a lot of fast unit tests, although I'm real quick to jettison the idea that a unit test only covers one single object under test, and if the result isn't "really" a unit test, I'll let the philosophers sort it out.
- 4y ago

Gabriel_h 4y ago

You may find this interesting too https://meticulous.ai/blog/let-users-write-tests-for-you/ https://meticulous.ai/blog/let-users-write-tests-for-you/ r.e. e2e testing and automatically generating mocks for UI tests. Maintaining mocks in e2e tests often becomes infeasible with a meaningful amount of e2e tests.

paulddraper 4y ago

Accurate. Unit tests are valuable if you're writing something with a very well defined interface like a base64 encoder. But bugs usually happen in the "glue". Static checks and e2e pound for pound catch the most bugs with the least effort.