11 ms·

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably usin

If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, you're probably using mocking wrong. If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at worst. Some APIs are very stable, and testing against the API itself can hit rate limiting, bans, and other anti-abuse mechanisms that introduce all kinds of instability to your test suite. Use the right tools to solve your problems.

Fire-Dragon-DoL 2y ago
100% agree
arcticbull 2y ago
When you write tests with mocks you almost always at some point end up with tests that test your mocks lol, and tests that test that you wrote the tests you think you wrote -- not the software itself. I’ve never been thrilled by tests that rely on mocking — it usually means you need to re-express your module interface boundary. Mocks for me fall into the class of software I affectionately call “load-bearing paint.” It’s basically universally the wrong tool for any given job but that really doesn’t stop people. Putting in a data class or similar model object and a delegate is usually sufficient and a much better tool.
- mehagar 2y ago
  I agree that if you need to write mocks, it's likely that your interfaces are poorly defined. This is one of the claimed benefits of test driven development - writing the tests first forces you to design the code in a way that cleanly separates modules so they can be tested.
  - UK-Al05 2y ago
    You have to mock/fake when modules call dependencies. Your way means you only ever have siblings. With an orchestrator pulling results out of one module and pushing it into another.
    - d0mine 2y ago
      Code that uses hexagonal architecture/dependency inversion requires less mocks in their tests.
      - UK-Al05 2y ago
        I use that architecture. You still use mocks at the edges.
      - thiht 2y ago
        That’s… not true? No matter how you define your dependencies to inject, if you want to mock the dependencies you inject you have to mock them (it’s almost tautological), no matter if you use dependency inversion or not Maybe you mean "less surface to mock", which is irrelevant if you generate your mocks automatically from the interface
        d0mine 2y ago
        > We can say that a Mock is a kind of spy, a spy is a kind of stub, and a stub is a kind of dummy. But a fake isn’t a kind of any of them. It’s a completely different kind of test double. You define fakes in this case, not mocks https://blog.cleancoder.com/uncle-bob/2014/05/14/TheLittleMocker.html https://blog.cleancoder.com/uncle-bob/2014/05/14/TheLittleMo...
        thiht 2y ago
        If you want. Replace mock by fake in my post and you get the same thing. It means I’m using fakes too when I’m not doing dependency inversion, so no mocks either For what it’s worth, I find the distinction between mocks, fakes, spies, stubs, dummies and whatever completely useless in practice, the whole point is to control some data flows to test in isolation, it’s really all that matters. Fun thing this kind of bikeshedding comes from Bob Martin, who famously has nothing worthwhile to show as an example of his actual work. Every time I read something from him, I get more and more convinced he’s a total fraud.
- xienze 2y ago
  > It’s basically universally the wrong tool for any given job but that really doesn’t stop people. I find mocks useful for testing conditions that are on the periphery and would be a decent amount of trouble to set up. For instance, if I have a REST controller that has a catch all for exceptions that maps everything to a 500 response, I want a test that will cause the DAO layer to throw an exception and test that the rest of the "real stack" will do the translation correctly. A mock is the easiest way to accomplish that.
  - chillpenguin 2y ago
    I agree. I will mock 3rd party APIs sometimes so I can test that my system correctly handles failures. For example, what if I get a 500 response from the API? With my mock I can easily make that happen. If I was using the actual API, I would have no way of forcing a 500 to happen.
- timcobb 2y ago
  > and a delegate And then you mock the delegate?
_hl_ 2y ago
Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up a new postgres DB for each unit test? - maintain this, eg have good, representative testing data lying around?
- didiskaka 2y ago
  > operate this? Do you spin up a new postgres DB for each unit test? Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". The big thing is to wrap each of your tests in a transaction which gets rolled back after each test. > maintain this, eg have good, representative testing data lying around This is much harder. Maintaining good seed data - data that covers all the edge cases - is a large amount of work. It's generally easier to leave it up to each test to setup data specific to their test case, generalizing that data when possible (i.e if you're testing login endpoints, you have all your login test cases inherit from some logic specific data setup, and they can tweak as needed from there). You will end up with duplicated test setup logic. It's not that bad, and often you don't really want to DRY this data anyways. That being said, if you have the time and resources to maintain seed data it's absolutely a better way to go about it. It's also beneficial outside of tests.
  - JasonSage 2y ago
    > Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". Every place I've ever worked which tried this has managed to get a production database deleted by somebody running tests.
    - MajimasEyepatch 2y ago
      It also wouldn't fly in many regulated environments.
hot_gril 2y ago
Also, don't create any interfaces you wouldn't mock. I've seen too many people waste months creating some kind of "database wrapper."
h1fra 2y ago
Clearly mocking DB is a footgun and it's not that hard to setup e2e test. Use TestContainer or Docker on a random port, run your API on a random port. Every tests seeds all the data needed to run (user, org, token), it requires an initial setup but then you just reuse it everywhere, and voila. No side effects, no mock to maintain, it also test your auth and permissions, almost 1:1 with prod.
- JackFr 2y ago
  > No side effects, no mock to maintain, it also test your auth and permissions, almost 1:1 with prod. Can also be used to test version updates of your DB.
gnfedhjmm2 2y ago
[flagged]
wongarsu 2y ago
And if you have to mock, at least try to have somebody else write the mock. Testing your understanding of GitHub's API against your understanding of GitHub's API isn't useful. Testing your interpretation of the API behavior against somebody else's interpretation provides a lot more value, even if it isn't nearly as good as testing against the actual API.
whack 2y ago
> If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at worst. Some APIs are very stable, and testing against the API itself can hit rate limiting, bans, and other anti-abuse mechanisms that introduce all kinds of instability to your test suite. I've been doing E2E testing using 3rd-party APIs for a decade now, and this has yet to be a significant problem. The majority of my APIs had a dedicated sandbox environment to avoid "rate limiting, bans, and other anti-abuse mechanisms". The remainder were simple enough that the provider didn't care about users exploring on the live API, and were usually read-only as well. Did I run into the occasional flaky failure, or API stability issues? Sure. But it was very rare and easy to workaround. It never devolved into becoming "untestable" or "full manual QA" My other teams that relied on mocks suffered from far worse problems - a ton of time being spent on manual-QA, and bugs that leaked into production, because of mock-reality mismatches.
- chii 2y ago
  > because of mock-reality mismatches. you also need to test the mocks against the real thing separately.
  - switchbak 2y ago
    I would prefer that we all try to use this language consistently: https://www.martinfowler.com/articles/mocksArentStubs.html https://www.martinfowler.com/articles/mocksArentStubs.html What you're describing sounds like a fake to me.
    - drewcoo 2y ago
      Can't tell from the context that's here. It's important to have a contract testing layer in place to make sure your test doubles are still behaving like the real thing, though.
      - ffsm8 2y ago
Pxtl 2y ago
The problem with the CRUD situations is I haven't seen a database that resets to precondition states quickly.
- timsehn 2y ago
  We built a MySQL-compatible one and are building a Postgres-compatible one :-) Free and open source. https://github.com/dolthub/dolt https://github.com/dolthub/dolt We use the reset functionality to speed up our tests. https://www.dolthub.com/blog/2022-06-10-enginetest-perf/ https://www.dolthub.com/blog/2022-06-10-enginetest-perf/
d0mine 2y ago
To test against real 3rd-party http API from time to time, and to generate mocks automatically for the same tests, you could use VCR https://pypi.org/project/vcrpy/ https://pypi.org/project/vcrpy/
koonsolo 2y ago
You could still use stubs instead of mocks in those cases.
0xbadcafebee 2y ago
> If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, Actually that's wrong too. The production database will be different than the "testing Postgres instance", leading to bugs. It turns out that whatever testing solution you use, if it's not the actual production instance and you're not using real production data, there will be bugs. Even then there's still bugs. This is the simple truth: you can't catch all the bugs. Just put in Good Enough testing for what you're doing and what you need, and get on with life. Otherwise you will spend 99% of your time just on testing.
- thiht 2y ago
  > The production database will be different than the "testing Postgres instance", leading to bugs. It never happened to me to be honest. This reads an argument for "if you can’t do perfect, just do it badly" but it’s nonsense. Running tests against a local Postgres instance with the same major.minor version and same extensions as your prod instance WILL work. And testing your storage layer against the database is probably the most reliable safety net you can add to an app.
  - 0xbadcafebee 2y ago
    > Running tests against a local Postgres instance with the same major.minor version and same extensions as your prod instance WILL work. A team I worked with recently said the same thing. But, as I predicted, they ran into bugs because the CloudSQL Postgres was different than their Dockerized Postgres, even though it was the same core version. There will always be testing problems you can't anticipate. Especially with systems that are not your own code. Just be ready to adapt your testing when it doesn't work as expected, and don't invest too much in the testing if it's not worth it.
    - thiht 2y ago
      Which bugs? I’ve been ready for 5 years as it’s the duration I’ve been testing my storage layers with 90% integration tests (the 10% being the hard to reproduce error cases, these tests have low value but are easy to test with mocks so I still test them). The only issue I’ve encountered was with time zones (shocking), and it made me ensure I got full control of the time zones in my app, in my deployment, in my local Postgres and in my prod Postgres, so net benefit in the end.
fire_lake 2y ago
Yes! Docker is a wonderful tool for testing.
stult 2y ago
One of the nice things about the .NET ORM EntityFramework is that you can swap a mocked in-memory database for your prod DB with dependency injection, so without modifying your code at all and theoretically without affecting the behavior of the ORM. Which is to say, you're right, it's about using the right tools. Those tools of course vary by ecosystem and so in some cases mocking the database is in fact the correct decision.
- bunderbunder 2y ago
  Probably the single most obnoxious production defect I ever found related to a database would never have made it into production if we had been using a real database instead of a test double. It happened because the test double failed to replicate a key detail in the database's transaction isolation rules. After figuring it out, I swapped us over to running all the tests that hit the database against the real database, in a testcontainer, with a RAM disk for minimizing query latency. It was about a day's worth of work, and turned up a few other bugs that hadn't bit us in production yet, too. Also sailing past our test suite because the test double failed to accurately replicate the behavior in question. Total time to run CI went up by about 10 seconds. (For local development you could chop that way down by not starting a fresh server instance for every test run.) Given how many person-hours we spent on diagnosing, resolving, and cleaning up after just that first defect, I estimated the nominally slower non-mocked tests are still a net time saver if amortized over anything less than about 50,000 CI runs, and even then we should probably only count the ones where an engineer is actually blocking on waiting for the tests to complete. That said, there was a time when I thought test doubles for databases was the most practical option because testing against real databases while maintaining test isolation was an unholy PITA. But that time was 5 or 6 years ago, before I had really learned how to use Docker properly.
  - hot_gril 2y ago
    I already don't trust ORMs; ORM with fake DB sounds even worse.
  - stult 2y ago
    I simply don't think that I will ever be able to come up with anything even vaguely as comprehensive as the test coverage that Microsoft already has for ensuring their ORM behaves consistently across database providers. In my over 10 years of using EF, I have never once encountered a database bug like you describe. If I were to discover such a bug (which I'll admit does occasionally happen even though it hasn't happened to me), it would be easier and better by far to submit an issue to the EF team and let them figure out a fix (including the appropriate tests) than it would be to rework my own test infrastructure. I am not in the business of developing requirements or code for databases, and building an elaborate test model for what I consider the essential requirements for a database would be a distraction from developing code that is more valuable to the business. The same logic does not apply to all ORMs, of course, which do not all benefit from the same quality of professional dev support that EF receives from MS. But that's my main point from above: the correct design decision depends on the context. For services written in other languages with other ORMs or raw SQL, I absolutely will spin up a full Postgres test container because it is indeed trivial (have one running in the background on my laptop right now in fact). It just isn't necessary in the specific context of EntityFramework code.
yellowapple 2y ago
> Some APIs are very stable, and testing against the API itself can hit rate limiting, bans, and other anti-abuse mechanisms that introduce all kinds of instability to your test suite. Those rate limits, bans, and other anti-abuse mechanisms are things that would be good to uncover and account for during tests. Better for the test suite to detect those potential failures than the production deployment :)

Fire-Dragon-DoL 2y ago

100% agree

arcticbull 2y ago

When you write tests with mocks you almost always at some point end up with tests that test your mocks lol, and tests that test that you wrote the tests you think you wrote -- not the software itself. I’ve never been thrilled by tests that rely on mocking — it usually means you need to re-express your module interface boundary. Mocks for me fall into the class of software I affectionately call “load-bearing paint.” It’s basically universally the wrong tool for any given job but that really doesn’t stop people. Putting in a data class or similar model object and a delegate is usually sufficient and a much better tool.

mehagar 2y ago
I agree that if you need to write mocks, it's likely that your interfaces are poorly defined. This is one of the claimed benefits of test driven development - writing the tests first forces you to design the code in a way that cleanly separates modules so they can be tested.
- UK-Al05 2y ago
  You have to mock/fake when modules call dependencies. Your way means you only ever have siblings. With an orchestrator pulling results out of one module and pushing it into another.
  - d0mine 2y ago
    Code that uses hexagonal architecture/dependency inversion requires less mocks in their tests.
    - UK-Al05 2y ago
      I use that architecture. You still use mocks at the edges.
    - thiht 2y ago
      That’s… not true? No matter how you define your dependencies to inject, if you want to mock the dependencies you inject you have to mock them (it’s almost tautological), no matter if you use dependency inversion or not Maybe you mean "less surface to mock", which is irrelevant if you generate your mocks automatically from the interface
      - d0mine 2y ago
        > We can say that a Mock is a kind of spy, a spy is a kind of stub, and a stub is a kind of dummy. But a fake isn’t a kind of any of them. It’s a completely different kind of test double. You define fakes in this case, not mocks https://blog.cleancoder.com/uncle-bob/2014/05/14/TheLittleMocker.html https://blog.cleancoder.com/uncle-bob/2014/05/14/TheLittleMo...
        thiht 2y ago
        If you want. Replace mock by fake in my post and you get the same thing. It means I’m using fakes too when I’m not doing dependency inversion, so no mocks either For what it’s worth, I find the distinction between mocks, fakes, spies, stubs, dummies and whatever completely useless in practice, the whole point is to control some data flows to test in isolation, it’s really all that matters. Fun thing this kind of bikeshedding comes from Bob Martin, who famously has nothing worthwhile to show as an example of his actual work. Every time I read something from him, I get more and more convinced he’s a total fraud.
xienze 2y ago
> It’s basically universally the wrong tool for any given job but that really doesn’t stop people. I find mocks useful for testing conditions that are on the periphery and would be a decent amount of trouble to set up. For instance, if I have a REST controller that has a catch all for exceptions that maps everything to a 500 response, I want a test that will cause the DAO layer to throw an exception and test that the rest of the "real stack" will do the translation correctly. A mock is the easiest way to accomplish that.
- chillpenguin 2y ago
  I agree. I will mock 3rd party APIs sometimes so I can test that my system correctly handles failures. For example, what if I get a 500 response from the API? With my mock I can easily make that happen. If I was using the actual API, I would have no way of forcing a 500 to happen.
timcobb 2y ago
> and a delegate And then you mock the delegate?

_hl_ 2y ago

Re. postgres, this is actually something I have always struggled with, so would love to learn how others do it. I’ve only ever worked in very small teams, where we didn’t really have the resources to maintain nice developer experiences and testing infrastructure. Even just maintaining representative testing data to seed a test DB as schemas (rapidly) evolve has been hard. So how do you - operate this? Do you spin up a new postgres DB for each unit test? - maintain this, eg have good, representative testing data lying around?

didiskaka 2y ago
> operate this? Do you spin up a new postgres DB for each unit test? Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". The big thing is to wrap each of your tests in a transaction which gets rolled back after each test. > maintain this, eg have good, representative testing data lying around This is much harder. Maintaining good seed data - data that covers all the edge cases - is a large amount of work. It's generally easier to leave it up to each test to setup data specific to their test case, generalizing that data when possible (i.e if you're testing login endpoints, you have all your login test cases inherit from some logic specific data setup, and they can tweak as needed from there). You will end up with duplicated test setup logic. It's not that bad, and often you don't really want to DRY this data anyways. That being said, if you have the time and resources to maintain seed data it's absolutely a better way to go about it. It's also beneficial outside of tests.
- JasonSage 2y ago
  > Generally I've seen a new database (schema in other dbs?) in postgres that is for testing, i.e "development_test" vs "development". Every place I've ever worked which tried this has managed to get a production database deleted by somebody running tests.
  - MajimasEyepatch 2y ago
    It also wouldn't fly in many regulated environments.

hot_gril 2y ago

Also, don't create any interfaces you wouldn't mock. I've seen too many people waste months creating some kind of "database wrapper."

h1fra 2y ago

Clearly mocking DB is a footgun and it's not that hard to setup e2e test. Use TestContainer or Docker on a random port, run your API on a random port. Every tests seeds all the data needed to run (user, org, token), it requires an initial setup but then you just reuse it everywhere, and voila. No side effects, no mock to maintain, it also test your auth and permissions, almost 1:1 with prod.

JackFr 2y ago
> No side effects, no mock to maintain, it also test your auth and permissions, almost 1:1 with prod. Can also be used to test version updates of your DB.

gnfedhjmm2 2y ago

[flagged]

wongarsu 2y ago

And if you have to mock, at least try to have somebody else write the mock. Testing your understanding of GitHub's API against your understanding of GitHub's API isn't useful. Testing your interpretation of the API behavior against somebody else's interpretation provides a lot more value, even if it isn't nearly as good as testing against the actual API.

whack 2y ago

> If you're writing a custom frontend for GitHub using the GitHub API and don't bother writing a decent set of mocks for how you expect the GitHub API to behave, your app will quickly require either full manual QA at best or become untestable at worst. Some APIs are very stable, and testing against the API itself can hit rate limiting, bans, and other anti-abuse mechanisms that introduce all kinds of instability to your test suite. I've been doing E2E testing using 3rd-party APIs for a decade now, and this has yet to be a significant problem. The majority of my APIs had a dedicated sandbox environment to avoid "rate limiting, bans, and other anti-abuse mechanisms". The remainder were simple enough that the provider didn't care about users exploring on the live API, and were usually read-only as well. Did I run into the occasional flaky failure, or API stability issues? Sure. But it was very rare and easy to workaround. It never devolved into becoming "untestable" or "full manual QA" My other teams that relied on mocks suffered from far worse problems - a ton of time being spent on manual-QA, and bugs that leaked into production, because of mock-reality mismatches.

chii 2y ago
> because of mock-reality mismatches. you also need to test the mocks against the real thing separately.
- switchbak 2y ago
  I would prefer that we all try to use this language consistently: https://www.martinfowler.com/articles/mocksArentStubs.html https://www.martinfowler.com/articles/mocksArentStubs.html What you're describing sounds like a fake to me.
  - drewcoo 2y ago
    Can't tell from the context that's here. It's important to have a contract testing layer in place to make sure your test doubles are still behaving like the real thing, though.
    - ffsm8 2y ago

Pxtl 2y ago

The problem with the CRUD situations is I haven't seen a database that resets to precondition states quickly.

timsehn 2y ago
We built a MySQL-compatible one and are building a Postgres-compatible one :-) Free and open source. https://github.com/dolthub/dolt https://github.com/dolthub/dolt We use the reset functionality to speed up our tests. https://www.dolthub.com/blog/2022-06-10-enginetest-perf/ https://www.dolthub.com/blog/2022-06-10-enginetest-perf/

d0mine 2y ago

To test against real 3rd-party http API from time to time, and to generate mocks automatically for the same tests, you could use VCR https://pypi.org/project/vcrpy/ https://pypi.org/project/vcrpy/

koonsolo 2y ago

You could still use stubs instead of mocks in those cases.

0xbadcafebee 2y ago

> If you're writing a CRUD app and mocking your database calls instead of just starting an actual Postgres instance before running the tests, Actually that's wrong too. The production database will be different than the "testing Postgres instance", leading to bugs. It turns out that whatever testing solution you use, if it's not the actual production instance and you're not using real production data, there will be bugs. Even then there's still bugs. This is the simple truth: you can't catch all the bugs. Just put in Good Enough testing for what you're doing and what you need, and get on with life. Otherwise you will spend 99% of your time just on testing.

thiht 2y ago
> The production database will be different than the "testing Postgres instance", leading to bugs. It never happened to me to be honest. This reads an argument for "if you can’t do perfect, just do it badly" but it’s nonsense. Running tests against a local Postgres instance with the same major.minor version and same extensions as your prod instance WILL work. And testing your storage layer against the database is probably the most reliable safety net you can add to an app.
- 0xbadcafebee 2y ago
  > Running tests against a local Postgres instance with the same major.minor version and same extensions as your prod instance WILL work. A team I worked with recently said the same thing. But, as I predicted, they ran into bugs because the CloudSQL Postgres was different than their Dockerized Postgres, even though it was the same core version. There will always be testing problems you can't anticipate. Especially with systems that are not your own code. Just be ready to adapt your testing when it doesn't work as expected, and don't invest too much in the testing if it's not worth it.
  - thiht 2y ago
    Which bugs? I’ve been ready for 5 years as it’s the duration I’ve been testing my storage layers with 90% integration tests (the 10% being the hard to reproduce error cases, these tests have low value but are easy to test with mocks so I still test them). The only issue I’ve encountered was with time zones (shocking), and it made me ensure I got full control of the time zones in my app, in my deployment, in my local Postgres and in my prod Postgres, so net benefit in the end.

fire_lake 2y ago

Yes! Docker is a wonderful tool for testing.

stult 2y ago

One of the nice things about the .NET ORM EntityFramework is that you can swap a mocked in-memory database for your prod DB with dependency injection, so without modifying your code at all and theoretically without affecting the behavior of the ORM. Which is to say, you're right, it's about using the right tools. Those tools of course vary by ecosystem and so in some cases mocking the database is in fact the correct decision.

bunderbunder 2y ago
Probably the single most obnoxious production defect I ever found related to a database would never have made it into production if we had been using a real database instead of a test double. It happened because the test double failed to replicate a key detail in the database's transaction isolation rules. After figuring it out, I swapped us over to running all the tests that hit the database against the real database, in a testcontainer, with a RAM disk for minimizing query latency. It was about a day's worth of work, and turned up a few other bugs that hadn't bit us in production yet, too. Also sailing past our test suite because the test double failed to accurately replicate the behavior in question. Total time to run CI went up by about 10 seconds. (For local development you could chop that way down by not starting a fresh server instance for every test run.) Given how many person-hours we spent on diagnosing, resolving, and cleaning up after just that first defect, I estimated the nominally slower non-mocked tests are still a net time saver if amortized over anything less than about 50,000 CI runs, and even then we should probably only count the ones where an engineer is actually blocking on waiting for the tests to complete. That said, there was a time when I thought test doubles for databases was the most practical option because testing against real databases while maintaining test isolation was an unholy PITA. But that time was 5 or 6 years ago, before I had really learned how to use Docker properly.
- hot_gril 2y ago
  I already don't trust ORMs; ORM with fake DB sounds even worse.
- stult 2y ago
  I simply don't think that I will ever be able to come up with anything even vaguely as comprehensive as the test coverage that Microsoft already has for ensuring their ORM behaves consistently across database providers. In my over 10 years of using EF, I have never once encountered a database bug like you describe. If I were to discover such a bug (which I'll admit does occasionally happen even though it hasn't happened to me), it would be easier and better by far to submit an issue to the EF team and let them figure out a fix (including the appropriate tests) than it would be to rework my own test infrastructure. I am not in the business of developing requirements or code for databases, and building an elaborate test model for what I consider the essential requirements for a database would be a distraction from developing code that is more valuable to the business. The same logic does not apply to all ORMs, of course, which do not all benefit from the same quality of professional dev support that EF receives from MS. But that's my main point from above: the correct design decision depends on the context. For services written in other languages with other ORMs or raw SQL, I absolutely will spin up a full Postgres test container because it is indeed trivial (have one running in the background on my laptop right now in fact). It just isn't necessary in the specific context of EntityFramework code.

yellowapple 2y ago

> Some APIs are very stable, and testing against the API itself can hit rate limiting, bans, and other anti-abuse mechanisms that introduce all kinds of instability to your test suite. Those rate limits, bans, and other anti-abuse mechanisms are things that would be good to uncover and account for during tests. Better for the test suite to detect those potential failures than the production deployment :)