4 ms·

You are able to go without tests because you're writing entire apps on your own. You know what each file, class, and module is supposed to do, and how to test

You are able to go without tests because you're writing entire apps on your own. You know what each file, class, and module is supposed to do, and how to test it for regressions manually. If you're working on a large team with a large suite of applications, this isn't the case. Very often I'll need to make changes to code that I don't explicitly own, or that I've never seen before today. If this code doesn't have tests, this is a recipe for disaster. I don't really understand what this code is supposed to do. How do I know that my change didn't break any existing functionality? Is it obvious how or even possible to test the functionality in this file manually? Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that? It's also about iteration time. Automated tests allow to you check the correctness of every single change nearly instantaneously. I don't want to need switch to a browser and wait for a form to submit just to check that my code still works after a minor refactor, or to ensure that my typo was fixed successfully. Tests mean that code is much more likely to be refactored often, and will lead to much cleaner and easier to maintain codebases.

candiodari 8y ago
This argument, and most of the others at this same level, have a glaring error. They only justify integration tests. They do not justify unit tests (in nearly all cases). Unit tests wouldn't catch the examples you gave: > How do I know that my change didn't break any existing functionality? You don't. And with many unit tests ... you still don't. A software engineer with a few years experience will know this. > Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that? This especially, no unit test is ever going to catch. This behavior would depend on so many things: encryption, networking, scheduling, starvation, graceful error handling, retry logic ... Not something any amount of unit tests could reasonably verify. So this must be an argument against unit tests ? And yet, I don't think so. In practice, you see the opposite. These arguments are designed to force people to write unit tests, and actually used against integration tests (which are complex tests that require tweaking in intelligent ways every time you make a serious change). Of course integration tests and "instantaneous" are usually opposites. So I don't understand this argument. It only makes sense on the surface. It does not actually work on real large code projects. So I don't understand that you make this argument. And I especially do not understand that nearly every large company follows advice like this. I mean, I can come up with cynical reasons: "more pointless work with zero verifiable results, but nice "quantifiable" (if meaningless) numbers ! Of course we're in favor !". But really, I cannot come up with a coherent to explain the behavior seen in developers in a large company. Most statistics about unit tests make no sense. Coverage for instance, says nothing about how many edge cases you handle. Says nothing about many guarantees (like bounded and reasonable allocation for "weird" inputs, same with cpu usage, that it still works together with the OS and other daemons on the system it'll be installed on, ...). Since tests have no "real" purpose for software (they're code that doesn't run) number of tests is meaningless too. Number of lines of test code ... that's beyond useless. The bigger a piece of software, the more useless unit tests become and the more critical realistic integration tests become ... and the less likely you are to actually find them. And frankly, for small pieces of code, I find that specialized ways to "test" them that provide actual guarantees of correctness (like Coq) actually work. Unit tests ... never catch edge cases, not even with the best of developers. Of course, even Coq sort-of only makes sure you KNOW about every edge case and requires you to verify it's correctness before letting you pass, so bugs in your understanding of the problem will still be bugs. But at least you have a theoretical guarantee that a developer actually thought about every edge case. Also, it's pretty hard to use. The language could be clearer, but that's not really the problem. The problem is that Coq WILL show you that no matter how well you think you understand a problem, your understanding is not complete. Often in surprising, interesting ways.
- fredifrum 8y ago
  The SFTP file processor is actually a quintessential example of how unit tests can help new developers edit existing code. A unit test for the processor would mock out all of the SFTP fetching, or move that to a different class, and focus on only the core business logic of processing the file. The core logic could easily be unit tested, and changes could be made to the file processor without needing to replicate the entire SFTP environment in order to determine if there were regressions in the core business logic. The alternative is needing to spin up a test SFTP environment, or somehow do that mocking in my manual test, just in order to do something as simple as refactor the class or make a small business logic change. A unit test empowers any developer to make those changes, without needing much knowledge of the environment the code runs in.
  - candiodari 8y ago
    > without needing to replicate the entire SFTP environment in order to determine if there were regressions Jep. And then it doesn't actually work. Because it tries to read every file at the same time. Because the strings it passes in to resolve the server don't work. Because it never schedules the thread reading from the socket. Because it uses a 100 byte buffer (plenty for tests, after all). Because ... And even then, what you're describing is not a unit test. A unit tests tests a single unit of code. Just 1 function, nothing more, and mocks out everything else. So you can never test a refactor of code with a unit test. Because a refactor changes how different pieces of code work together, and unit tests by definition explicitly DON'T test that (all interactions should be mocked out). So a unit test would be useless.
    - fredifrum 8y ago
      The fact that something might go wrong in the integration test doesn't mean unit tests for the core logic aren't helpful. Besides, you're probably going to be using an external library for the SFTP connect, so it's very likely to go just fine. And you can totally use unit tests for what I'm describing. Two classes - SFTPHandler. Connects via SFTP, downloads latest files off the server, passes contents as a string to the processor class `FileProcessor.process(downloaded_file)` - FileProcessor. Has one public function, process, which processes the file - doing whatever it needs to. This function can then very easily be unit tested, just passing strings for test files into the function. You can also refactor the `process` function as much as you like, not needing to worry about the SFTP connection at all. The `process` function probably calls a bunch of private functions within that class, but your unit tests don't need to worry about that. I've used a setup like this in production, it works just fine, and allowed us to improve the performance of the file processing logic and make changes to it very easily and often - without worrying about regressions to the core business logic.

candiodari 8y ago

This argument, and most of the others at this same level, have a glaring error. They only justify integration tests. They do not justify unit tests (in nearly all cases). Unit tests wouldn't catch the examples you gave: > How do I know that my change didn't break any existing functionality? You don't. And with many unit tests ... you still don't. A software engineer with a few years experience will know this. > Maybe it's a background job that processes a file off of an SFTP server, how long will it take me to set up a manual test for that? This especially, no unit test is ever going to catch. This behavior would depend on so many things: encryption, networking, scheduling, starvation, graceful error handling, retry logic ... Not something any amount of unit tests could reasonably verify. So this must be an argument against unit tests ? And yet, I don't think so. In practice, you see the opposite. These arguments are designed to force people to write unit tests, and actually used against integration tests (which are complex tests that require tweaking in intelligent ways every time you make a serious change). Of course integration tests and "instantaneous" are usually opposites. So I don't understand this argument. It only makes sense on the surface. It does not actually work on real large code projects. So I don't understand that you make this argument. And I especially do not understand that nearly every large company follows advice like this. I mean, I can come up with cynical reasons: "more pointless work with zero verifiable results, but nice "quantifiable" (if meaningless) numbers ! Of course we're in favor !". But really, I cannot come up with a coherent to explain the behavior seen in developers in a large company. Most statistics about unit tests make no sense. Coverage for instance, says nothing about how many edge cases you handle. Says nothing about many guarantees (like bounded and reasonable allocation for "weird" inputs, same with cpu usage, that it still works together with the OS and other daemons on the system it'll be installed on, ...). Since tests have no "real" purpose for software (they're code that doesn't run) number of tests is meaningless too. Number of lines of test code ... that's beyond useless. The bigger a piece of software, the more useless unit tests become and the more critical realistic integration tests become ... and the less likely you are to actually find them. And frankly, for small pieces of code, I find that specialized ways to "test" them that provide actual guarantees of correctness (like Coq) actually work. Unit tests ... never catch edge cases, not even with the best of developers. Of course, even Coq sort-of only makes sure you KNOW about every edge case and requires you to verify it's correctness before letting you pass, so bugs in your understanding of the problem will still be bugs. But at least you have a theoretical guarantee that a developer actually thought about every edge case. Also, it's pretty hard to use. The language could be clearer, but that's not really the problem. The problem is that Coq WILL show you that no matter how well you think you understand a problem, your understanding is not complete. Often in surprising, interesting ways.

fredifrum 8y ago
The SFTP file processor is actually a quintessential example of how unit tests can help new developers edit existing code. A unit test for the processor would mock out all of the SFTP fetching, or move that to a different class, and focus on only the core business logic of processing the file. The core logic could easily be unit tested, and changes could be made to the file processor without needing to replicate the entire SFTP environment in order to determine if there were regressions in the core business logic. The alternative is needing to spin up a test SFTP environment, or somehow do that mocking in my manual test, just in order to do something as simple as refactor the class or make a small business logic change. A unit test empowers any developer to make those changes, without needing much knowledge of the environment the code runs in.
- candiodari 8y ago
  > without needing to replicate the entire SFTP environment in order to determine if there were regressions Jep. And then it doesn't actually work. Because it tries to read every file at the same time. Because the strings it passes in to resolve the server don't work. Because it never schedules the thread reading from the socket. Because it uses a 100 byte buffer (plenty for tests, after all). Because ... And even then, what you're describing is not a unit test. A unit tests tests a single unit of code. Just 1 function, nothing more, and mocks out everything else. So you can never test a refactor of code with a unit test. Because a refactor changes how different pieces of code work together, and unit tests by definition explicitly DON'T test that (all interactions should be mocked out). So a unit test would be useless.
  - fredifrum 8y ago
    The fact that something might go wrong in the integration test doesn't mean unit tests for the core logic aren't helpful. Besides, you're probably going to be using an external library for the SFTP connect, so it's very likely to go just fine. And you can totally use unit tests for what I'm describing. Two classes - SFTPHandler. Connects via SFTP, downloads latest files off the server, passes contents as a string to the processor class `FileProcessor.process(downloaded_file)` - FileProcessor. Has one public function, process, which processes the file - doing whatever it needs to. This function can then very easily be unit tested, just passing strings for test files into the function. You can also refactor the `process` function as much as you like, not needing to worry about the SFTP connection at all. The `process` function probably calls a bunch of private functions within that class, but your unit tests don't need to worry about that. I've used a setup like this in production, it works just fine, and allowed us to improve the performance of the file processing logic and make changes to it very easily and often - without worrying about regressions to the core business logic.