5 ms·

The SFTP file processor is actually a quintessential example of how unit tests can help new developers edit existing code. A unit test for the processor would

The SFTP file processor is actually a quintessential example of how unit tests can help new developers edit existing code. A unit test for the processor would mock out all of the SFTP fetching, or move that to a different class, and focus on only the core business logic of processing the file. The core logic could easily be unit tested, and changes could be made to the file processor without needing to replicate the entire SFTP environment in order to determine if there were regressions in the core business logic. The alternative is needing to spin up a test SFTP environment, or somehow do that mocking in my manual test, just in order to do something as simple as refactor the class or make a small business logic change. A unit test empowers any developer to make those changes, without needing much knowledge of the environment the code runs in.

candiodari 8y ago
> without needing to replicate the entire SFTP environment in order to determine if there were regressions Jep. And then it doesn't actually work. Because it tries to read every file at the same time. Because the strings it passes in to resolve the server don't work. Because it never schedules the thread reading from the socket. Because it uses a 100 byte buffer (plenty for tests, after all). Because ... And even then, what you're describing is not a unit test. A unit tests tests a single unit of code. Just 1 function, nothing more, and mocks out everything else. So you can never test a refactor of code with a unit test. Because a refactor changes how different pieces of code work together, and unit tests by definition explicitly DON'T test that (all interactions should be mocked out). So a unit test would be useless.
- fredifrum 8y ago
  The fact that something might go wrong in the integration test doesn't mean unit tests for the core logic aren't helpful. Besides, you're probably going to be using an external library for the SFTP connect, so it's very likely to go just fine. And you can totally use unit tests for what I'm describing. Two classes - SFTPHandler. Connects via SFTP, downloads latest files off the server, passes contents as a string to the processor class `FileProcessor.process(downloaded_file)` - FileProcessor. Has one public function, process, which processes the file - doing whatever it needs to. This function can then very easily be unit tested, just passing strings for test files into the function. You can also refactor the `process` function as much as you like, not needing to worry about the SFTP connection at all. The `process` function probably calls a bunch of private functions within that class, but your unit tests don't need to worry about that. I've used a setup like this in production, it works just fine, and allowed us to improve the performance of the file processing logic and make changes to it very easily and often - without worrying about regressions to the core business logic.
  - candiodari 8y ago
    In my experience if there's one thing absolutely guaranteed it's that unit tests decrease the performance of whatever they're testing (Because it eventually leads to local wins that are big losses for the program as a whole, because this encourages doing idiotic stuff like allocating large buffers and keeping enormous non-shared caches and maps) Now in the example given, performance does not matter, so I do wonder why you'd mention it at all. How about you just answer me this question: Did you still see significant bug volumes after implementing the unit tests for the FileProcessor ? Obviously I believe the answer to be "yes". I feel like your statement that changes were made "very easily and often" sort of implies that yes, there were many bugs. Note that testing based on past bugs is not called unit testing. That is, as you might guess, regression testing (and has the important distinction that it's a VERY good practice to go back through your regression tests once a year, and throw out the ones that don't make sense anymore, which should be about half of them) Besides, I've very rarely seen tests actually catch bugs. Bugs come from pieces of code not doing what developers expect them to do in 2 ways : 1) outright lack of understanding what the code does (this can also mean that they understand the code, but not the problem it's trying to solve, and so code and tests ... are simply both wrong) 2) lack of consideration for edge cases 3) lack of consideration for the environment the code runs in (e.g. scaling issues. Optimizing business logic that processes a 10M file then executing it on 50G of data) None of these has a good chance of getting caught by unit tests in my experience. But developers seem to mostly hate integration tests. Tests that start up the whole system, or even multiple copies of it, and then rapidly run past input through the whole system. When it fails, it takes a while to find why it fails. It may fail, despite all components, potentially written by different people, being "correctly written" just not taking each other into account. It may fail because of memory, cpu starvation, filesystem setup. It may fail occassionally because the backend database decided to VACUUM, and the app is not backing off. It may fail after a firmware upgrade on equipment it uses. The problem I have with these "issues" is simple: they represent reality. They will occur in production. And in some ways I feel like this is a fair description: unit tests are about "proving you're right", even, and perhaps especially, if you're wrong. "You see, it isn't my code ! Not my fault !".
    - fredifrum 8y ago

candiodari 8y ago

> without needing to replicate the entire SFTP environment in order to determine if there were regressions Jep. And then it doesn't actually work. Because it tries to read every file at the same time. Because the strings it passes in to resolve the server don't work. Because it never schedules the thread reading from the socket. Because it uses a 100 byte buffer (plenty for tests, after all). Because ... And even then, what you're describing is not a unit test. A unit tests tests a single unit of code. Just 1 function, nothing more, and mocks out everything else. So you can never test a refactor of code with a unit test. Because a refactor changes how different pieces of code work together, and unit tests by definition explicitly DON'T test that (all interactions should be mocked out). So a unit test would be useless.

fredifrum 8y ago
The fact that something might go wrong in the integration test doesn't mean unit tests for the core logic aren't helpful. Besides, you're probably going to be using an external library for the SFTP connect, so it's very likely to go just fine. And you can totally use unit tests for what I'm describing. Two classes - SFTPHandler. Connects via SFTP, downloads latest files off the server, passes contents as a string to the processor class `FileProcessor.process(downloaded_file)` - FileProcessor. Has one public function, process, which processes the file - doing whatever it needs to. This function can then very easily be unit tested, just passing strings for test files into the function. You can also refactor the `process` function as much as you like, not needing to worry about the SFTP connection at all. The `process` function probably calls a bunch of private functions within that class, but your unit tests don't need to worry about that. I've used a setup like this in production, it works just fine, and allowed us to improve the performance of the file processing logic and make changes to it very easily and often - without worrying about regressions to the core business logic.
- candiodari 8y ago
  In my experience if there's one thing absolutely guaranteed it's that unit tests decrease the performance of whatever they're testing (Because it eventually leads to local wins that are big losses for the program as a whole, because this encourages doing idiotic stuff like allocating large buffers and keeping enormous non-shared caches and maps) Now in the example given, performance does not matter, so I do wonder why you'd mention it at all. How about you just answer me this question: Did you still see significant bug volumes after implementing the unit tests for the FileProcessor ? Obviously I believe the answer to be "yes". I feel like your statement that changes were made "very easily and often" sort of implies that yes, there were many bugs. Note that testing based on past bugs is not called unit testing. That is, as you might guess, regression testing (and has the important distinction that it's a VERY good practice to go back through your regression tests once a year, and throw out the ones that don't make sense anymore, which should be about half of them) Besides, I've very rarely seen tests actually catch bugs. Bugs come from pieces of code not doing what developers expect them to do in 2 ways : 1) outright lack of understanding what the code does (this can also mean that they understand the code, but not the problem it's trying to solve, and so code and tests ... are simply both wrong) 2) lack of consideration for edge cases 3) lack of consideration for the environment the code runs in (e.g. scaling issues. Optimizing business logic that processes a 10M file then executing it on 50G of data) None of these has a good chance of getting caught by unit tests in my experience. But developers seem to mostly hate integration tests. Tests that start up the whole system, or even multiple copies of it, and then rapidly run past input through the whole system. When it fails, it takes a while to find why it fails. It may fail, despite all components, potentially written by different people, being "correctly written" just not taking each other into account. It may fail because of memory, cpu starvation, filesystem setup. It may fail occassionally because the backend database decided to VACUUM, and the app is not backing off. It may fail after a firmware upgrade on equipment it uses. The problem I have with these "issues" is simple: they represent reality. They will occur in production. And in some ways I feel like this is a fair description: unit tests are about "proving you're right", even, and perhaps especially, if you're wrong. "You see, it isn't my code ! Not my fault !".
  - fredifrum 8y ago