10 ms·

Zen and the Art of Unit Testing

150 points by Liriel 10y ago

royjacobs 10y ago
I see this type of mock-heavy testing in lots of places. I used to do the same thing too, but I found that it basically just tightly couples your test to your implementation details. The whole purpose of unit tests is to be able to refactor things with confidence that your tests will still pass. If you so closely tie your test to your implementation you lose that. In the example, the tests now really care about how data is being retrieved from the database. Instead, I'd use an in-memory database (or even a Dockerized one). That would then also test that the right queries are being done, but you could still refactor the internals of UserServiceImpl (terrible name btw) and your tests wouldn't fail.
- scarface74 10y ago
  I completely agree with the idea of an "in memory database". I use Entity Framework. I mock the EF context to return regular in memory Lists of objects in unit tests the linq -> sql translation is replace with linq to objects.
- lubonay 10y ago
  I tend to do mock-heavy testing in Java projects, and it has burned me several times. You write mock tests along your architectural boundaries, but if those need to change due to a deeper refactoring, all your tests need to get refactored as well. Mock testing essentially tests expected side effects. A more powerful concept is the use of pure functions wherever possible, so that your tests compare input/output pairs, instead of long, difficult to maintain and sometimes non-exhaustive lists of expected side effects. Does anybody know how one can replace Services, Controllers, Presenters and other such imperative mediator objects with a more functional approach? I'm just speculating, but that should make test maintenance easier.
  - beaconstudios 10y ago
    I find that you can keep with a class-based approach if you utilise your IOC container in your tests, and only swap out side-effectful classes (ORM, remote service clients, etc) with mocks. That way you don't need to concern yourself with the dependencies of the classes you're testing so you can refactor the underlying relationships and objects with impunity as long as you don't change the interfaces of the classes you're testing.
    - lubonay 10y ago
      I don't know if I asked my question as clearly as possible. Mocking your DB, the file system, the remote API, etc are all a given when doing tests. My question was aimed at trying to find out if it's feasible to replace the imperative Presenter/Service/etc patterns with a more functional approach. What I'd like to see is testing only the returned value of a function, as opposed to calling a void function and checking whether the injected mocks got the proper method invocations, i.e. whether the desired side effects occurred.
      - beaconstudios 10y ago
        if you're calling a void function then by definition you're calling a function that does nothing but side effects or calling off to other classes. In the latter case, they can be good candidates for refactoring in my experience. A function that does something but doesn't either produce mockable side-effects or return information may be a code smell.
        cytzol 10y ago
        A function can return void and still throw an exception. (Like one I wrote today, "insert row into table", which throws on failure and return nothing on success.) A bunch of functions that all return void all being called in a row, though? That's a much smellier smell. What happens if their order gets mixed up, for example.
        beaconstudios 10y ago
  - greydius 10y ago
    You might be interested in the "free monad" approach used occasionally in scala and haskell. It separates the effectful operations from the effects by first building a data structure that describes what to do and then interpreting that data structure. The key is that you can interpret the data structure in different ways. That is, one interpreter for production and another for testing. The advantage this has over mocking is that the business logic becomes a collection of pure functions.
  - adamkl 10y ago
    Take a look at Gary Bernhardt's talk "Boundaries"[1]. It touches on this very topic and was quite the eye-opener for me (having experienced all the same issues you've described). [1]https://www.destroyallsoftware.com/talks/boundaries https://www.destroyallsoftware.com/talks/boundaries
    - lubonay 10y ago
      I think this may hit the nail on the head, gonna check it out later. Thanks!
  - lmm 10y ago
    Monads. Possibly the Free Monad. Make your business logic pure logic that computes commands, and separate the computation of the command from its execution, then you can use a different execution to test your command chains. http://michaelxavier.net/posts/2014-04-27-Cool-Idea-Free-Monads-for-Testing-Redis-Calls.html http://michaelxavier.net/posts/2014-04-27-Cool-Idea-Free-Mon... has a basic example, but it works well for replacing your services/controllers/what-have-you too.
  - arwhatever 10y ago
    Although it's a pay site, may I highly recommend https://app.pluralsight.com/library/courses/csharp-applying-functional-principles/table-of-contents https://app.pluralsight.com/library/courses/csharp-applying-... Also check out that author's (Vladimir Khorikov) blog.
- UK-AL 10y ago
  "tests now really care about how data is being retrieved from the database". I'm not sure this correct this case. He mocks the repo. The interface for the repo cares about the intention, the implementation cares about the "how". If your creating a user, you always going to want a "add" to persistent store intention. You don't really care how the add to persistent store is done though, or what persistent store. You just care the service intended to do so. I have an idea that you if your performing a command, you should mock. The whole points of a command is to coordinate subsystems to perform an action. So you need to check if the right methods were called. If your performing a query you should stub or in memory db.
- zamalek 10y ago
  > I found that it basically just tightly couples your test to your implementation details Compilation in strictest form of TDD is the first failing test that you run. Catching this has advantages, such as knowledge and contemplation of altered ABI. Your interface refactoring could also originate from your tests. Alter the code in your test (even the mock interface) until it is clean at that usage site, and the compile/test run until everything passes.
  - hamilyon2 10y ago
    This is undesirable in my book. Mere refactoring should just check all boxes in test suite. If I changed tests as well, how would I be sure no functionality changed? Tests did not serve their's purpouse in the end
    - adrianratnapala 10y ago
      This whole discussion has been about an "interface refactoring". I.e. a change to internal interfaces so obviously anything that sees that interface is subject to change. Now it is true that some interfaces are more stable than others, and you can make a good argument that testing shouldn't be done at boundaries that are likely to be unstable.
- galephico 10y ago
  how would you call UserServiceImpl? Underlying question, how would you organize your CRUD (with business logic) operations?
  - royjacobs 10y ago
    I think I would reverse the question. Why does there have to be a UserService interface? If it only exists in order to be mocked in yet another place of the code you've added even more unnecessary layering in your codebase.
- lojack 10y ago
  > The whole purpose of unit tests is to be able to refactor things with confidence that your tests will still pass. (assuming the second part of that statement should have been "that your program will still be correct") I'm back and forth on my agreement/disagreement on this one. One of the major purposes of tests is to have confidence in refactoring, but it isn't the _only_ purpose. Beyond that though, I'd say that unit tests probably aren't the right tool for the job when we're talking about refactoring. The purpose of unit tests is to verify that the unit of code is working in isolation. Integration tests are much better suited for verifying correctness when refactoring. Any good test suite should have a healthy balance of unit tests and integration tests. And, when I say integration tests, I mean whatever you want to call them -- tests that check the system as a whole. I don't necessarily disagree with you though. Mocks aren't perfect, and they have their own host of problems. I've run into a number of passing tests that should have never been that way because of mocks. It happens. The best way to combat this is to use mocks more sparingly, and have feature tests for many of these cases.
- stephen 10y ago
  Agreed. A test should care about: input, "do something", output. Then you can refactor the heck out of "do something" and your test (which is ideally documenting the use cases you need to support) doesn't have to change at all. For me, preferring stubs (dummy implementations of your external dependencies), when possible/appropriate, is a good way to avoid "your test is just a repetition of which method calls your implementation makes": http://www.draconianoverlord.com/2010/07/09/why-i-dont-like-mocks.html http://www.draconianoverlord.com/2010/07/09/why-i-dont-like-...
tener 10y ago
> Without DI you are forced to write slow and difficult to maintain integration tests instead of unit tests. Integration tests are worthwhile even if you do have unit tests. They serve different purposes.
- UK-AL 10y ago
  Yes, but you keep them separate from unit tests. Unit tests should be fast to run, for quick feedback.
latch 10y ago
I've been writing tests for a long time, and I went through a phase where I depended heavily on DI. While it's still useful in some conditions, I rarely make use of it. First, DI is simply the least subtle for of IoC available. When all else fails, you can always rely on DI. But, this is something languages should be looking to help developers with. Elixir, for example, supports IoC as part of the language, and is much more elegant than DI [1]. When monkey patching is available, this is also a possible solution (easy to abuse, yes, but suitable in simple cases). Functions as first class values can also help (instead of passing a clock around, you can have a SystemClock.Time func that you can override in your tests, which is similar to what Date.now() is in javascript). But, perhaps more importantly, and as much as I tried to deny it, integration tests are absolutely and totally worth the trouble. If you unit test two parts independently, you run the very real risk of having them pass under test, but fail once deployed. I've seen more production downtime caused by incorrect assumptions between services than anything else. Also, lately, I've been writing more and more fuzz tests. I'm probably not very good at it yet, but I think for the couple of projects we did it, it's been a worthwhile effort (moreso when we started...they barely catch anything now since we're all coding much more defensively). [1] http://openmymind.net/Dependency-Injection-In-Elixir/ http://openmymind.net/Dependency-Injection-In-Elixir/
- UK-AL 10y ago
  Nobody denies integration tests are good. You just don't want your unit tests to be integration tests. Unit tests are meant to be fast, and quick to isolate the failing code.
  - snovv_crash 10y ago
    If you make your so-called-'unit' tests too closely coupled to you don't actually work faster though, because you constantly have to refactor them to match the internal implementation. Your testing feedback might be quicker, but quickly getting useless information (ie. your implementation changed) is just as bad as slower tests, because you need to fix the test, recompile and again before you get useful information. Also not all 'integration tests' are slow. If you can test using SQlite in memory, instead of pointing at a PG instance that needs to be provisioned, for instance, you can get a lot of your end-to-end testing running really quickly.
stuartd 10y ago
Misleading title - there is absolutely no Zen in this article. The nearest it gets is a quote from Confucius, who wasn't a Buddhist.
- ChefDenominator 10y ago
  I wish people would stop using that title. They seem to think it is a clever click-grabber, which is probably true for most people, but for those even remotely familiar with the book, Zen and the Art of Motorcycle Maintenance: An Inquiry into Values, opening up the article and finding no application of Pirsig's Metaphysics of Quality is disappointing.
  - tclancy 10y ago
    And what is clickbait, Phaedrus, And what is not clickbait— Need we ask anyone to tell us these things?
  - Insanity 10y ago
    And I might add, the book is worth reading!
  - _pmf_ 10y ago
    To be fair, I found the Zen in ZatAoMM to also be more pretense (whether this was intended by Pirsig or not)..
  - stuartd 10y ago
    The title of Pirsig's book is itself a riff on "Zen in the Art of Archery", published in 1948.
  - abiox 10y ago
    there's plenty of such things... "X considered harmful" seemed to have an uptick in usage for a while.
    - alberto_ol 10y ago
      also "what we talk about whwn we talk about...", "the unreasonable effectiveness of ..."
sidlls 10y ago
I must be one of the few who think this sort of unit-testing is necessary to a certain extent, but in practice seems rather to lend itself to causing terrible code smells all over the place: things like small one-liner functions for conditionals and basic arithmetic that are designed with mock- and test-ability in mind (rather than whether or not they make the code maintainable or correct), factoring code specifically to increase "code coverage" of the tests, etc. I think integration and other coarse-grained tests are far more useful more often, and using asserts (which will cause these coarse-grained tests to fail, too) is a better pattern than "unit test all the things".
- jwilliams 10y ago
  I've seen this in practice too. Lots of unit tests for small things that actually make the overall codebase brittle. I think it comes from a pretty decent practice of coding from the tests outward. However I'd argue the long term value of many of those tests is minimal.
  - adrianratnapala 10y ago
    Yes. I write lots of unit tests, but they are mostly part of the coding process: I find they are superior to the manual testing I would have to do otherwise every time I add new code. But as long term protections against regressions, such tests are less valuable than more coarse-grained integration tests.
- deleted 10y ago
  [deleted]
- cytzol 10y ago
  > things like small one-liner functions for conditionals and basic arithmetic that are designed with mock- and test-ability in mind I've done this before. It's very easy to want to make all your function/method calls work the same. To stop, I applied the following rule whenever I had to choose between making a function static or mockable: does this function's name completely describe its implementation? Small methods like "StringUtils::isEmptyOrWhitespace" pass this test: it's just going to check if the string is empty or only contains whitespace, and that doesn't need to have its implementation swapped out. On the other side, "ImageUtils::detectFacesInImage" goes into no detail about how it does the face detection, and definitely needs an interface to test against. That said, it all depends. You might allow your users to pick which characters count as whitespace, I don't know. > I think integration and other coarse-grained tests are far more useful more often! Yes! Integration tests are the "main" tests. Unit tests are secondary. Use them when you need to debug why your integration tests fail.
_pmf_ 10y ago
> startResetPasswordProcess_givenEmailOfExistingUser_sendsNotificationToUser It's sad that this is the state of the art when unit testing (but I have not found any viable alternative when the premise is that I have to use JUnit).
macca321 10y ago
It's a shame that the functional requirements, which were neatly stated in the spec, weren't preserved in the test code. Rather than "arrange", "act", "assert", I've taken to writing the Gherkin spec inline with the test code, e.g. @Test public void onlyExistingEmailsCanResetTheirPasswords() { //Given an email address which is not in the database String email = "unknown@example.com"; when(userRepository.findByEmailAddress("unknown@example.com")) .thenReturn(null); //When a password reset is requested for that email UserServiceImpl userService = ... userService.startResetPasswordProcess("unknown@example.com"); //Then no password reset notification should be sent verify(notificationService, never()).sendResetPasswordNotification(any()); //And the user should be given a success message so they can't determine if the email address is already present verify(userInterface, once()).sendMessage("You have been sent Please check your email."); } This way you preserve the intent for the programmers that will come after you, but you don't need a complicated Gherkin runner
jrpelkonen 10y ago
> String newPasswordHash = cryptoService.sha1(newPassword); Ouch. It's 2017. I know it's a blog about unit testing, but using SHA-1, let alone unsalted SHA-1, for password hashing even if for illustration purposes is dangerous.
- mdaniel 10y ago
  At the risk of spitting into the wind, so is using String in Java for passwords; the actual password-centric interfaces in the JVM use char[] because one can zero-out those, but cannot zero-out immutable Strings
mybrid 10y ago
I found the art of testing is more tied to the reality on the ground as opposed to the testing methodology. 1. Do you have a a QA department doing manual or even automated testing? 2. Do you have performance testing? 3. Is the code base large or small? 4. Transactions versus analytics? 5. Social media? Some code doesn't need testing. Facebook doesn't test their code. Each new project requires a flexibility to understand the reality of time and resources available for testing, the type of application and performance considerations. If one works at a bank or on transactional software then testing needs to verify exactness. If you are working on analytical software then what's another anomaly between friends, errors in your errors, who cares. If you are working on social media then just pump and dump like NodeJS. That is the art of testing to me. A mind set of one size fits all with respect to the of value unit testing, integration testing, regression testing or manual testing is the opposite of art: it's ideology.
mybrid 10y ago
Understand the workload. Take the workload difference between in house versus public usage. If one releases software into the public domain then one has no reasonable expectation as to the kinds of data driving your code. For this kind of general purpose, heavily reused software then code coverage with, unit testing and mocking makes the most sense is my experience. However, one can also write code for a very specific, limited usage. In those cases I find regression testing with regression data works best. They are not paying me to find bugs to satisfy code coverage. I find understanding the workload is a significant factor in determining the kinds of testing I use.
mybrid 10y ago
Testing cost money. I find it very unusual to talk with developers who regularly modify their testing practice based on time and money. The most expensive is to do it all: unit testing, integration testing, component testing, system testing, end-to-end testing, stochastic testing, performance testing. The art of testing in no small part comes down to the art of money. How much bang-for-the buck does any testing provide given limited resources? I answer that question.

150 points by Liriel 10y ago

I see this type of mock-heavy testing in lots of places. I used to do the same thing too, but I found that it basically just tightly couples your test to your implementation details. The whole purpose of unit tests is to be able to refactor things with confidence that your tests will still pass. If you so closely tie your test to your implementation you lose that. In the example, the tests now really care about how data is being retrieved from the database. Instead, I'd use an in-memory database (or even a Dockerized one). That would then also test that the right queries are being done, but you could still refactor the internals of UserServiceImpl (terrible name btw) and your tests wouldn't fail.

scarface74 10y ago
I completely agree with the idea of an "in memory database". I use Entity Framework. I mock the EF context to return regular in memory Lists of objects in unit tests the linq -> sql translation is replace with linq to objects.
lubonay 10y ago
I tend to do mock-heavy testing in Java projects, and it has burned me several times. You write mock tests along your architectural boundaries, but if those need to change due to a deeper refactoring, all your tests need to get refactored as well. Mock testing essentially tests expected side effects. A more powerful concept is the use of pure functions wherever possible, so that your tests compare input/output pairs, instead of long, difficult to maintain and sometimes non-exhaustive lists of expected side effects. Does anybody know how one can replace Services, Controllers, Presenters and other such imperative mediator objects with a more functional approach? I'm just speculating, but that should make test maintenance easier.
- beaconstudios 10y ago
  I find that you can keep with a class-based approach if you utilise your IOC container in your tests, and only swap out side-effectful classes (ORM, remote service clients, etc) with mocks. That way you don't need to concern yourself with the dependencies of the classes you're testing so you can refactor the underlying relationships and objects with impunity as long as you don't change the interfaces of the classes you're testing.
  - lubonay 10y ago
    I don't know if I asked my question as clearly as possible. Mocking your DB, the file system, the remote API, etc are all a given when doing tests. My question was aimed at trying to find out if it's feasible to replace the imperative Presenter/Service/etc patterns with a more functional approach. What I'd like to see is testing only the returned value of a function, as opposed to calling a void function and checking whether the injected mocks got the proper method invocations, i.e. whether the desired side effects occurred.
    - beaconstudios 10y ago
      if you're calling a void function then by definition you're calling a function that does nothing but side effects or calling off to other classes. In the latter case, they can be good candidates for refactoring in my experience. A function that does something but doesn't either produce mockable side-effects or return information may be a code smell.
      - cytzol 10y ago
        A function can return void and still throw an exception. (Like one I wrote today, "insert row into table", which throws on failure and return nothing on success.) A bunch of functions that all return void all being called in a row, though? That's a much smellier smell. What happens if their order gets mixed up, for example.
        beaconstudios 10y ago
- greydius 10y ago
  You might be interested in the "free monad" approach used occasionally in scala and haskell. It separates the effectful operations from the effects by first building a data structure that describes what to do and then interpreting that data structure. The key is that you can interpret the data structure in different ways. That is, one interpreter for production and another for testing. The advantage this has over mocking is that the business logic becomes a collection of pure functions.
- adamkl 10y ago
  Take a look at Gary Bernhardt's talk "Boundaries"[1]. It touches on this very topic and was quite the eye-opener for me (having experienced all the same issues you've described). [1]https://www.destroyallsoftware.com/talks/boundaries https://www.destroyallsoftware.com/talks/boundaries
  - lubonay 10y ago
    I think this may hit the nail on the head, gonna check it out later. Thanks!
- lmm 10y ago
  Monads. Possibly the Free Monad. Make your business logic pure logic that computes commands, and separate the computation of the command from its execution, then you can use a different execution to test your command chains. http://michaelxavier.net/posts/2014-04-27-Cool-Idea-Free-Monads-for-Testing-Redis-Calls.html http://michaelxavier.net/posts/2014-04-27-Cool-Idea-Free-Mon... has a basic example, but it works well for replacing your services/controllers/what-have-you too.
- arwhatever 10y ago
  Although it's a pay site, may I highly recommend https://app.pluralsight.com/library/courses/csharp-applying-functional-principles/table-of-contents https://app.pluralsight.com/library/courses/csharp-applying-... Also check out that author's (Vladimir Khorikov) blog.
UK-AL 10y ago
"tests now really care about how data is being retrieved from the database". I'm not sure this correct this case. He mocks the repo. The interface for the repo cares about the intention, the implementation cares about the "how". If your creating a user, you always going to want a "add" to persistent store intention. You don't really care how the add to persistent store is done though, or what persistent store. You just care the service intended to do so. I have an idea that you if your performing a command, you should mock. The whole points of a command is to coordinate subsystems to perform an action. So you need to check if the right methods were called. If your performing a query you should stub or in memory db.
zamalek 10y ago
> I found that it basically just tightly couples your test to your implementation details Compilation in strictest form of TDD is the first failing test that you run. Catching this has advantages, such as knowledge and contemplation of altered ABI. Your interface refactoring could also originate from your tests. Alter the code in your test (even the mock interface) until it is clean at that usage site, and the compile/test run until everything passes.
- hamilyon2 10y ago
  This is undesirable in my book. Mere refactoring should just check all boxes in test suite. If I changed tests as well, how would I be sure no functionality changed? Tests did not serve their's purpouse in the end
  - adrianratnapala 10y ago
    This whole discussion has been about an "interface refactoring". I.e. a change to internal interfaces so obviously anything that sees that interface is subject to change. Now it is true that some interfaces are more stable than others, and you can make a good argument that testing shouldn't be done at boundaries that are likely to be unstable.
galephico 10y ago
how would you call UserServiceImpl? Underlying question, how would you organize your CRUD (with business logic) operations?
- royjacobs 10y ago
  I think I would reverse the question. Why does there have to be a UserService interface? If it only exists in order to be mocked in yet another place of the code you've added even more unnecessary layering in your codebase.
lojack 10y ago
> The whole purpose of unit tests is to be able to refactor things with confidence that your tests will still pass. (assuming the second part of that statement should have been "that your program will still be correct") I'm back and forth on my agreement/disagreement on this one. One of the major purposes of tests is to have confidence in refactoring, but it isn't the _only_ purpose. Beyond that though, I'd say that unit tests probably aren't the right tool for the job when we're talking about refactoring. The purpose of unit tests is to verify that the unit of code is working in isolation. Integration tests are much better suited for verifying correctness when refactoring. Any good test suite should have a healthy balance of unit tests and integration tests. And, when I say integration tests, I mean whatever you want to call them -- tests that check the system as a whole. I don't necessarily disagree with you though. Mocks aren't perfect, and they have their own host of problems. I've run into a number of passing tests that should have never been that way because of mocks. It happens. The best way to combat this is to use mocks more sparingly, and have feature tests for many of these cases.
stephen 10y ago
Agreed. A test should care about: input, "do something", output. Then you can refactor the heck out of "do something" and your test (which is ideally documenting the use cases you need to support) doesn't have to change at all. For me, preferring stubs (dummy implementations of your external dependencies), when possible/appropriate, is a good way to avoid "your test is just a repetition of which method calls your implementation makes": http://www.draconianoverlord.com/2010/07/09/why-i-dont-like-mocks.html http://www.draconianoverlord.com/2010/07/09/why-i-dont-like-...

tener 10y ago

> Without DI you are forced to write slow and difficult to maintain integration tests instead of unit tests. Integration tests are worthwhile even if you do have unit tests. They serve different purposes.

UK-AL 10y ago
Yes, but you keep them separate from unit tests. Unit tests should be fast to run, for quick feedback.

latch 10y ago

I've been writing tests for a long time, and I went through a phase where I depended heavily on DI. While it's still useful in some conditions, I rarely make use of it. First, DI is simply the least subtle for of IoC available. When all else fails, you can always rely on DI. But, this is something languages should be looking to help developers with. Elixir, for example, supports IoC as part of the language, and is much more elegant than DI [1]. When monkey patching is available, this is also a possible solution (easy to abuse, yes, but suitable in simple cases). Functions as first class values can also help (instead of passing a clock around, you can have a SystemClock.Time func that you can override in your tests, which is similar to what Date.now() is in javascript). But, perhaps more importantly, and as much as I tried to deny it, integration tests are absolutely and totally worth the trouble. If you unit test two parts independently, you run the very real risk of having them pass under test, but fail once deployed. I've seen more production downtime caused by incorrect assumptions between services than anything else. Also, lately, I've been writing more and more fuzz tests. I'm probably not very good at it yet, but I think for the couple of projects we did it, it's been a worthwhile effort (moreso when we started...they barely catch anything now since we're all coding much more defensively). [1] http://openmymind.net/Dependency-Injection-In-Elixir/ http://openmymind.net/Dependency-Injection-In-Elixir/

UK-AL 10y ago
Nobody denies integration tests are good. You just don't want your unit tests to be integration tests. Unit tests are meant to be fast, and quick to isolate the failing code.
- snovv_crash 10y ago
  If you make your so-called-'unit' tests too closely coupled to you don't actually work faster though, because you constantly have to refactor them to match the internal implementation. Your testing feedback might be quicker, but quickly getting useless information (ie. your implementation changed) is just as bad as slower tests, because you need to fix the test, recompile and again before you get useful information. Also not all 'integration tests' are slow. If you can test using SQlite in memory, instead of pointing at a PG instance that needs to be provisioned, for instance, you can get a lot of your end-to-end testing running really quickly.

stuartd 10y ago

Misleading title - there is absolutely no Zen in this article. The nearest it gets is a quote from Confucius, who wasn't a Buddhist.

ChefDenominator 10y ago
I wish people would stop using that title. They seem to think it is a clever click-grabber, which is probably true for most people, but for those even remotely familiar with the book, Zen and the Art of Motorcycle Maintenance: An Inquiry into Values, opening up the article and finding no application of Pirsig's Metaphysics of Quality is disappointing.
- tclancy 10y ago
  And what is clickbait, Phaedrus, And what is not clickbait— Need we ask anyone to tell us these things?
- Insanity 10y ago
  And I might add, the book is worth reading!
- _pmf_ 10y ago
  To be fair, I found the Zen in ZatAoMM to also be more pretense (whether this was intended by Pirsig or not)..
- stuartd 10y ago
  The title of Pirsig's book is itself a riff on "Zen in the Art of Archery", published in 1948.
- abiox 10y ago
  there's plenty of such things... "X considered harmful" seemed to have an uptick in usage for a while.
  - alberto_ol 10y ago
    also "what we talk about whwn we talk about...", "the unreasonable effectiveness of ..."

sidlls 10y ago

I must be one of the few who think this sort of unit-testing is necessary to a certain extent, but in practice seems rather to lend itself to causing terrible code smells all over the place: things like small one-liner functions for conditionals and basic arithmetic that are designed with mock- and test-ability in mind (rather than whether or not they make the code maintainable or correct), factoring code specifically to increase "code coverage" of the tests, etc. I think integration and other coarse-grained tests are far more useful more often, and using asserts (which will cause these coarse-grained tests to fail, too) is a better pattern than "unit test all the things".

jwilliams 10y ago
I've seen this in practice too. Lots of unit tests for small things that actually make the overall codebase brittle. I think it comes from a pretty decent practice of coding from the tests outward. However I'd argue the long term value of many of those tests is minimal.
- adrianratnapala 10y ago
  Yes. I write lots of unit tests, but they are mostly part of the coding process: I find they are superior to the manual testing I would have to do otherwise every time I add new code. But as long term protections against regressions, such tests are less valuable than more coarse-grained integration tests.
deleted 10y ago
[deleted]
cytzol 10y ago
> things like small one-liner functions for conditionals and basic arithmetic that are designed with mock- and test-ability in mind I've done this before. It's very easy to want to make all your function/method calls work the same. To stop, I applied the following rule whenever I had to choose between making a function static or mockable: does this function's name completely describe its implementation? Small methods like "StringUtils::isEmptyOrWhitespace" pass this test: it's just going to check if the string is empty or only contains whitespace, and that doesn't need to have its implementation swapped out. On the other side, "ImageUtils::detectFacesInImage" goes into no detail about how it does the face detection, and definitely needs an interface to test against. That said, it all depends. You might allow your users to pick which characters count as whitespace, I don't know. > I think integration and other coarse-grained tests are far more useful more often! Yes! Integration tests are the "main" tests. Unit tests are secondary. Use them when you need to debug why your integration tests fail.

_pmf_ 10y ago

> startResetPasswordProcess_givenEmailOfExistingUser_sendsNotificationToUser It's sad that this is the state of the art when unit testing (but I have not found any viable alternative when the premise is that I have to use JUnit).

macca321 10y ago

It's a shame that the functional requirements, which were neatly stated in the spec, weren't preserved in the test code. Rather than "arrange", "act", "assert", I've taken to writing the Gherkin spec inline with the test code, e.g. @Test public void onlyExistingEmailsCanResetTheirPasswords() { //Given an email address which is not in the database String email = "unknown@example.com"; when(userRepository.findByEmailAddress("unknown@example.com")) .thenReturn(null); //When a password reset is requested for that email UserServiceImpl userService = ... userService.startResetPasswordProcess("unknown@example.com"); //Then no password reset notification should be sent verify(notificationService, never()).sendResetPasswordNotification(any()); //And the user should be given a success message so they can't determine if the email address is already present verify(userInterface, once()).sendMessage("You have been sent Please check your email."); } This way you preserve the intent for the programmers that will come after you, but you don't need a complicated Gherkin runner

jrpelkonen 10y ago

> String newPasswordHash = cryptoService.sha1(newPassword); Ouch. It's 2017. I know it's a blog about unit testing, but using SHA-1, let alone unsalted SHA-1, for password hashing even if for illustration purposes is dangerous.

mdaniel 10y ago
At the risk of spitting into the wind, so is using String in Java for passwords; the actual password-centric interfaces in the JVM use char[] because one can zero-out those, but cannot zero-out immutable Strings

mybrid 10y ago

I found the art of testing is more tied to the reality on the ground as opposed to the testing methodology. 1. Do you have a a QA department doing manual or even automated testing? 2. Do you have performance testing? 3. Is the code base large or small? 4. Transactions versus analytics? 5. Social media? Some code doesn't need testing. Facebook doesn't test their code. Each new project requires a flexibility to understand the reality of time and resources available for testing, the type of application and performance considerations. If one works at a bank or on transactional software then testing needs to verify exactness. If you are working on analytical software then what's another anomaly between friends, errors in your errors, who cares. If you are working on social media then just pump and dump like NodeJS. That is the art of testing to me. A mind set of one size fits all with respect to the of value unit testing, integration testing, regression testing or manual testing is the opposite of art: it's ideology.

mybrid 10y ago

Understand the workload. Take the workload difference between in house versus public usage. If one releases software into the public domain then one has no reasonable expectation as to the kinds of data driving your code. For this kind of general purpose, heavily reused software then code coverage with, unit testing and mocking makes the most sense is my experience. However, one can also write code for a very specific, limited usage. In those cases I find regression testing with regression data works best. They are not paying me to find bugs to satisfy code coverage. I find understanding the workload is a significant factor in determining the kinds of testing I use.

mybrid 10y ago

Testing cost money. I find it very unusual to talk with developers who regularly modify their testing practice based on time and money. The most expensive is to do it all: unit testing, integration testing, component testing, system testing, end-to-end testing, stochastic testing, performance testing. The art of testing in no small part comes down to the art of money. How much bang-for-the buck does any testing provide given limited resources? I answer that question.