5 ms·

The whole mocks, fakes and the rest were always a terrible idea. Although people didn't realize it st the time, they were a band-aid to try to make poorly archi

The whole mocks, fakes and the rest were always a terrible idea. Although people didn't realize it st the time, they were a band-aid to try to make poorly architected code testable. 90% of the time, needing to use a mock is one of the clearest code warning smells you have of there being an issue in the design your code. It took a while, but the industry seems to be finally (although slowly) coming to this realization. And hopefully with it almost all of this can go away.

danparsonson 9mo ago
How do you substitute for dependencies that you're not testing, or that you want to deliberately break?
- bluGill 9mo ago
  90% of the time (or more): you don't. The real thing is perfectly fine in a test with the right setup. Fileio is fast, I just need a test file in a tempdir. databases are fast, I just need an easy way to settup my schema. Sometimes I need the isolation but normally I do not.
  - asa400 9mo ago
    I worked on a project where a dev wanted to mock out the database in tests "for performance, because the database is slow". I almost lost my shit. Even funnier, this was all hypothetical and yet taken as gospel. We hadn't even written the tests yet, so it was impossible to say whether they were slow or not. Nothing had been measured, no performance budget had been defined, no prototype of the supposedly slow tests had been written to demonstrate the point. We ended up writing - no joke - less than 100 tests total, almost all of which hit the database, including some full integration tests, and the entire test suite finished in a few seconds. I'm all for building in a way that respects performance as an engineering value, but we got lost somewhere along the way.
    - danparsonson 9mo ago
      > I almost lost my shit. Hopefully one day you'll back at that, and realise what an immature attitude that was.
      - asa400 9mo ago
        I'm sorry you felt the need to post this in response to a figure of speech. Please keep your moralizing to yourself. Thank you.
        danparsonson 9mo ago
        Well of course it was a figure of speech - if you meant it literally then I would not by prodding you about your medical condition. I was posting in response to your story that a colleague proposed a way of working or technical solution to something, and that this in some way enraged or otherwise upset you. And then you thought that made a good story to share online. That is, as I said, an immature response to a disagreement. Unless that whole paragraph was a figure of speech, in which case what I said doesn't apply, and we can both go about our days.
        asa400 9mo ago
        No thanks, I'll pass on the amateur psychoanalysis. Good luck to you sir.
      - wiseowise 9mo ago
        Hopefully one day I won’t have to deal with clowns like you.
        danparsonson 9mo ago
        Aw don't say that, you'll make me sad! There's nothing worse than a sad clown.
    - 8note 9mo ago
      tne knly reason i like having mocks os so you can define all the data objects the whole way through, and see the whole workflow as if you'd written it on paper, without having to open a bunch of files or jump around. just top to bottom every object that is supposed to be there in what order
  - danparsonson 9mo ago
    Well, no - you don't. What you're describing is a very limited subset of testing, which presumably is fine for the projects you work on, but that experience does not generalise well. Integration testing is of course useful, but generally one would want to create unit tests for every part of the code, and by definition it's not a unit test if hits multiple parts of the code simultaneously. Apart from that, databases and file access may be fast but they still take resources and time to spin up; beyond a certain project and team size, it's far cheaper to mock those things. With a mock you can also easily simulate failure cases, bad data, etc. - how do you test for file access issues, or the database server being offline? Using mocks properly is a sign of a well-factored codebase.
    - bluGill 9mo ago
      Why would I want to create tests for every part of the code? I did that for years because I was taught that, but I came to realize it never mattered - if a test breaks it is because of the last thing I changed. I have a few flakey tests from time to time, but they have been not too bad to track down nd often taught me enough about how the system really worked as to be worth the time anyway.
      - danparsonson 9mo ago
        I'm sorry, I don't understand what you mean. You seem to say it's not worth writing a lot of tests, but then you talk about tests breaking due to bad changes - if you don't write those tests in the first place, then how do you get into that situation? I didn't word my earlier comment very well - I don't mean to advocate for 100% coverage, which I personally think is a waste of time at best, and a false comfort at worst. Is this what you're talking about? What I wanted to say is that unit tests should be written for every part of the code that you're testing, i.e. break it into bits rather than test the whole thing in one lump, or better, do both - unit tests and integration tests.
        bluGill 9mo ago
        Write a lot of tests - mostly integration. Unit tests have proven more harmful than helpful - unit tests are great when the api is used so often it would be painful to change it so you don't anyway. Otherwise I want to change the api and the code that uses it as requirements change. When I'm writing string or a list I'd unit test that - but mostly that is in my standard library so I'm not. Instead I'm writing code that is only used a few places and those places both will change every few years as requirements change.
  - 8note 9mo ago
    how are you deliberately breaking those dependencies? or are you only testing the happy path? you could extend this to say 85% of the tome just write the code directly to prod and dont have any tests. if you broke something, an alarm will go off
  - bccdee 9mo ago
    What if you're calling an API? How do you test a tool that does a bunch of back-and-forth with another service without actually hitting that other service? Do you have to spin up your entire k8s ecosystem in a local cluster just to validate that the logic in one function is sound? Do you have to deliberately misconfigure it in order to test that your function handles errors properly? More broadly, suppose foo() has an implementation that depends on Bar, but Bar is complicated to instantiate because it needs to know about 5 external services. Fortunately foo() only depends on a narrow sliver of Bar's functionality. Why not wrap Bar in a narrow interface—only the bits foo() depends on—and fake it? I'm not a maximalist about test doubles. I prefer to factor out my I/O until it's high-level enough that it doesn't need unit tests. But that's not always an option, and I'd rather be flexible and use a test double than burden all my unit tests with the full weight of their production dependencies.
    - bluGill 9mo ago
      I have never done a full k8s but I have started dbus on a non-standard port for isolation.
- mangodrunk 9mo ago
  Why substitute dependencies? Is the isolation worth it?
  - danparsonson 9mo ago
    For the same reason you isolate variables in a scientific experiment; to ensure you're controlling the test that you're running, and not accidentally testing something else. To easily simulate failure cases, a range of possible inputs, bad data etc. To make the testing process faster when you have hundreds or thousands of tests, running on multiple builds simultaneously across an organisation. Off the top of my head :-)
    - mangodrunk 9mo ago
      I don’t think it’s worth doing that, and comparing it to scientific experiments doesn’t really apply. You can do all that without mocks as well. Making the tests run faster at the expense of better tests seems counterproductive. Now you should think of reasons why you should not isolate.
      - danparsonson 9mo ago
        > I don’t think it’s worth doing that OK; it's your choice to do what you think is right. > and comparing it to scientific experiments doesn’t really apply. Why not? I think it's a fairly apt comparison; you have a theory ("this piece of code does the following things"), and write tests to prove it. > You can do all that without mocks as well. OK, but mocks make it easier and cleaner - so why wouldn't I do that? > Making the tests run faster at the expense of better tests seems counterproductive. Smaller, more focused, cleaner tests are better in my opinion; speed is a beneficial side effect. > Now you should think of reasons why you should not isolate. Why? That's your argument - it's not on me to prove it for you. If you can give me some good reason why mocking out the interfaces you are not testing is a bad idea, and some better alternative, then we can have a discussion about it.
        mangodrunk 9mo ago
mangodrunk 9mo ago
I agree. Tests relying on mocks rarely uncover or prevent issues. They also typically make it harder to make changes. Very bad idea that should have been left behind years ago.
fud101 9mo ago
Ok, can someone explain this to someone with double digit iq? Is it validating the adapter idea or something else?

danparsonson 9mo ago

How do you substitute for dependencies that you're not testing, or that you want to deliberately break?

bluGill 9mo ago
90% of the time (or more): you don't. The real thing is perfectly fine in a test with the right setup. Fileio is fast, I just need a test file in a tempdir. databases are fast, I just need an easy way to settup my schema. Sometimes I need the isolation but normally I do not.
- asa400 9mo ago
  I worked on a project where a dev wanted to mock out the database in tests "for performance, because the database is slow". I almost lost my shit. Even funnier, this was all hypothetical and yet taken as gospel. We hadn't even written the tests yet, so it was impossible to say whether they were slow or not. Nothing had been measured, no performance budget had been defined, no prototype of the supposedly slow tests had been written to demonstrate the point. We ended up writing - no joke - less than 100 tests total, almost all of which hit the database, including some full integration tests, and the entire test suite finished in a few seconds. I'm all for building in a way that respects performance as an engineering value, but we got lost somewhere along the way.
  - danparsonson 9mo ago
    > I almost lost my shit. Hopefully one day you'll back at that, and realise what an immature attitude that was.
    - asa400 9mo ago
      I'm sorry you felt the need to post this in response to a figure of speech. Please keep your moralizing to yourself. Thank you.
      - danparsonson 9mo ago
        Well of course it was a figure of speech - if you meant it literally then I would not by prodding you about your medical condition. I was posting in response to your story that a colleague proposed a way of working or technical solution to something, and that this in some way enraged or otherwise upset you. And then you thought that made a good story to share online. That is, as I said, an immature response to a disagreement. Unless that whole paragraph was a figure of speech, in which case what I said doesn't apply, and we can both go about our days.
        asa400 9mo ago
        No thanks, I'll pass on the amateur psychoanalysis. Good luck to you sir.
    - wiseowise 9mo ago
      Hopefully one day I won’t have to deal with clowns like you.
      - danparsonson 9mo ago
        Aw don't say that, you'll make me sad! There's nothing worse than a sad clown.
  - 8note 9mo ago
    tne knly reason i like having mocks os so you can define all the data objects the whole way through, and see the whole workflow as if you'd written it on paper, without having to open a bunch of files or jump around. just top to bottom every object that is supposed to be there in what order
- danparsonson 9mo ago
  Well, no - you don't. What you're describing is a very limited subset of testing, which presumably is fine for the projects you work on, but that experience does not generalise well. Integration testing is of course useful, but generally one would want to create unit tests for every part of the code, and by definition it's not a unit test if hits multiple parts of the code simultaneously. Apart from that, databases and file access may be fast but they still take resources and time to spin up; beyond a certain project and team size, it's far cheaper to mock those things. With a mock you can also easily simulate failure cases, bad data, etc. - how do you test for file access issues, or the database server being offline? Using mocks properly is a sign of a well-factored codebase.
  - bluGill 9mo ago
    Why would I want to create tests for every part of the code? I did that for years because I was taught that, but I came to realize it never mattered - if a test breaks it is because of the last thing I changed. I have a few flakey tests from time to time, but they have been not too bad to track down nd often taught me enough about how the system really worked as to be worth the time anyway.
    - danparsonson 9mo ago
      I'm sorry, I don't understand what you mean. You seem to say it's not worth writing a lot of tests, but then you talk about tests breaking due to bad changes - if you don't write those tests in the first place, then how do you get into that situation? I didn't word my earlier comment very well - I don't mean to advocate for 100% coverage, which I personally think is a waste of time at best, and a false comfort at worst. Is this what you're talking about? What I wanted to say is that unit tests should be written for every part of the code that you're testing, i.e. break it into bits rather than test the whole thing in one lump, or better, do both - unit tests and integration tests.
      - bluGill 9mo ago
        Write a lot of tests - mostly integration. Unit tests have proven more harmful than helpful - unit tests are great when the api is used so often it would be painful to change it so you don't anyway. Otherwise I want to change the api and the code that uses it as requirements change. When I'm writing string or a list I'd unit test that - but mostly that is in my standard library so I'm not. Instead I'm writing code that is only used a few places and those places both will change every few years as requirements change.
- 8note 9mo ago
  how are you deliberately breaking those dependencies? or are you only testing the happy path? you could extend this to say 85% of the tome just write the code directly to prod and dont have any tests. if you broke something, an alarm will go off
- bccdee 9mo ago
  What if you're calling an API? How do you test a tool that does a bunch of back-and-forth with another service without actually hitting that other service? Do you have to spin up your entire k8s ecosystem in a local cluster just to validate that the logic in one function is sound? Do you have to deliberately misconfigure it in order to test that your function handles errors properly? More broadly, suppose foo() has an implementation that depends on Bar, but Bar is complicated to instantiate because it needs to know about 5 external services. Fortunately foo() only depends on a narrow sliver of Bar's functionality. Why not wrap Bar in a narrow interface—only the bits foo() depends on—and fake it? I'm not a maximalist about test doubles. I prefer to factor out my I/O until it's high-level enough that it doesn't need unit tests. But that's not always an option, and I'd rather be flexible and use a test double than burden all my unit tests with the full weight of their production dependencies.
  - bluGill 9mo ago
    I have never done a full k8s but I have started dbus on a non-standard port for isolation.
mangodrunk 9mo ago
Why substitute dependencies? Is the isolation worth it?
- danparsonson 9mo ago
  For the same reason you isolate variables in a scientific experiment; to ensure you're controlling the test that you're running, and not accidentally testing something else. To easily simulate failure cases, a range of possible inputs, bad data etc. To make the testing process faster when you have hundreds or thousands of tests, running on multiple builds simultaneously across an organisation. Off the top of my head :-)
  - mangodrunk 9mo ago
    I don’t think it’s worth doing that, and comparing it to scientific experiments doesn’t really apply. You can do all that without mocks as well. Making the tests run faster at the expense of better tests seems counterproductive. Now you should think of reasons why you should not isolate.
    - danparsonson 9mo ago
      > I don’t think it’s worth doing that OK; it's your choice to do what you think is right. > and comparing it to scientific experiments doesn’t really apply. Why not? I think it's a fairly apt comparison; you have a theory ("this piece of code does the following things"), and write tests to prove it. > You can do all that without mocks as well. OK, but mocks make it easier and cleaner - so why wouldn't I do that? > Making the tests run faster at the expense of better tests seems counterproductive. Smaller, more focused, cleaner tests are better in my opinion; speed is a beneficial side effect. > Now you should think of reasons why you should not isolate. Why? That's your argument - it's not on me to prove it for you. If you can give me some good reason why mocking out the interfaces you are not testing is a bad idea, and some better alternative, then we can have a discussion about it.
      - mangodrunk 9mo ago

mangodrunk 9mo ago

I agree. Tests relying on mocks rarely uncover or prevent issues. They also typically make it harder to make changes. Very bad idea that should have been left behind years ago.

fud101 9mo ago

Ok, can someone explain this to someone with double digit iq? Is it validating the adapter idea or something else?