5 ms·

Don’t code with mocks period. Structure your code such that it has a functional core and imperative shell. All logic is unit testable (functional core). All IO

Don’t code with mocks period. Structure your code such that it has a functional core and imperative shell. All logic is unit testable (functional core). All IO and mutation Is not unit testable (imperative shell). Mocks come from a place where logic is heavily intertwined with IO. It means your code is so coupled that you can’t test logic without touching IO. IO should be a dumb layer as much as possible. It should be Extremely general and as simple as fetching and requesting in the stupidest way possible. Then everything else should be covered by unit tests while IO is fundamentally not unit testable. If you have a lot of complex sql statements or logic going on in the IO layer (for performance reasons don’t do this if you can’t help it) it means your IO layer needs to be treated like a logical layer. Make sure you have stored procedures and unit tests written in your database IO language. You code should interface only with stored procedures as IO and those stored procedures should be tested as unit tests in the IO layer.

ljm 2y ago
so what you should do instead is write a load of pure functions and intersperse them with impure functions, and when it comes to testing that, if someone writes a different impure function than the one you wanted, you just fail the test and trigger a post to HN: "don't code with mocks."
- ninetyninenine 2y ago
  You failed to read and/or understand what I wrote. Also how do I fail a test for a new code addition I didn’t even write? Unit tests dont test impure functions all they do is test pure functions. Impure functions aren’t unit tested period. So when someone writes an impure function I don’t like what happens? Nothing. It wasn’t part of unit tests annyway, understand? If you’re referring to integration tests well a newly written impure function isn’t part of integration tests anyway so nothing happens in this case either. I found your last comment about how I’m triggered to write such a comment rude and lacking in even understanding what I wrote. Please try to understand my point before saying something like that… if you can’t please refrain from replying in such a way as it’s against the rules here. Thanks for reading.
  - piva00 2y ago
    How do you write the expected return from an external API in your test setup?
    - ninetyninenine 2y ago
      That’s an integration test. Your external api returns a number, let’s say 10. You have two local functions. One that is very general and fetches that number, and another that transforms that number. Your local transformation function is the one that is tested. The other function is part of integration tests and in general can’t be unit tested.
      - piva00 2y ago
        Exactly, there's a layer (integration testing) where I usually do need to have a test against a mock to validate my function doing the external call is passing a request I expect. I can break it down into a parameter supplier function that will generate the proper request params and test that but I don't see any gain in abstracting that away simply to avoid mocks.
    - Spivak 2y ago
      You use a "VCR" module to record the real request and response and then play it back in your unit test. Boom you get the power of an integration test with the speed and determinism of a unit test. Then you turn off the VCR in CI to catch if/when the upstream API changes on you and now it's a real integration test. The problem with write tests, not too many, mostly integration is that unit tests are too damn good at giving you instant feedback and a tight dev loop. So we turned integration tests into unit tests. Combined with SQLite :memory: for unit tests and Postgres for the final integration tests we don't have to mock anything. Source: Currently use this method at $dayjob with great success.
      - piva00 2y ago
        My question was a rhetorical one, I do use request-replayers and other techniques (I've been through a bunch in the past 20+ years as they were developed). I was challenging the absolutism of "never use mocks", it's just another technique and can be applied easily in integration tests if the guidelines are well set in the team on how to not use them. Also, I do not do external calls in the CI/CD pipeline, it inevitably makes tests brittle and flaky.
throwawaymaths 2y ago
how do you test the imperative shell except with mocks? Suppose you're calling an external API
- tomjakubowski 2y ago
  The jargon around all this stuff is terrible. What I would probably write is, like, a fake version of that API, one which runs out of process from the library or test, and which can be "programmed" from the test set up -- "expect these requests, serve these responses", etc. Is that a mock? I don't know. It sure does resemble one, but it doesn't use dependency injection or other runtime tricks which I'd associate with classic "mocks"; it's just another program you write to support the tests. The imperative shell doing I/O needs to be configurable to support this (think a "widgets API base URL"), but that's it.
  - ninetyninenine 2y ago
    The jargon is terrible because I think you misunderstood my point. You don’t mock period is my philosophy but your talking about mocking an entire api here.
  - BoiledCabbage 2y ago
    Yes that still a mock. And thr point is to structure your code in such a way that there is little to no value left in testing that API call independently, because all of the related logic has been tested in a unit test. Once you've done that, you can just run a simple integration or end-to-end test to cover that API. You never need to manually mock it (like you suggested), or use a mocking tool.
    - drewcoo 2y ago
      > that still a mock It's a fake, not a mock. Mocks and fakes are both test doubles. > And thr point is to structure your code in such a way that Sometimes it's not your code. Sometimes you want to change legacy code and want tests to prove that your changes don't break anything. Life would be so much easier if there were just one way to test, wouldn't it?
paulryanrogers 2y ago
Please don't write a lot of stored procedures. Changing them in large or sharded DBs is not trivial and tooling to support them is minimal.
spawarotti 2y ago
Great advice. I follow it in my coding efforts and it has never failed me. Great book about this: Unit Testing Principles, Practices, and Patterns, Vladimir Khorikov, 2020 https://www.manning.com/books/unit-testing https://www.manning.com/books/unit-testing
dxxmxnd 2y ago
Any suggestions on where to read about this more?
- tokonoma 2y ago
  https://www.destroyallsoftware.com/talks/boundaries https://www.destroyallsoftware.com/talks/boundaries
  - cryptonector 2y ago
    That's the only site whose dev contents I ever paid for. Gary Bernhardt is very entertaining.
bcrosby95 2y ago
> IO should be a dumb layer as much as possible. It should be Extremely general and as simple as fetching and requesting in the stupidest way possible. Then everything else should be covered by unit tests while IO is fundamentally not unit testable. Another way to put this is to ignore all the cool problems your database can solve for you and hand roll it instead. This also is a recipe for riddling your codebase with data races. You're trading fairly easy problems for very hard problems. No thanks. When running tests, optimize your database configuration for running tests against it, and make it possible to run those tests in parallel.
- AndyNemmity 2y ago
  Right, but you can see how they do it. The problems they are trading are likely not hard for them. Whatever you use, in your personal toolbox changes the results of the tradeoffs.
scarface_74 2y ago
I am actually at a point in my career that I won’t work for a company that is dependent on stored procedures. Rollbacks, versioning, branching, testing and everything else is a pain once you introduce stored procedures. As a lead for mostly green field applications over the past decade, my one no compromise architectural edict is no stored procedures for online production code. I might use it for background utilities or one off maintenance.
- tbrownaw 2y ago
  Is this about having application logic in multiple places, or having application logic co-located with the data? Does it make a difference if the procedures are database triggers vs being directly called by the application code? Would splitting off some of the logic into a separate service (that does its own database accesses) have the same issues?
  - scarface_74 2y ago
    Say I have functionality to get customer data. If the sql logic is in code, I can create a new branch and work on it, merge my changes, I can do blue/green deployments. A rollback if you’re using Kubernetes is a simple matter of reverting back to the configuration file containing a reference to the old Docker container etc. How do you roll back a dozen stored procedures quickly? How do you just create a new branch or test a regression? With all of the logic in code, it’s a simple matter of checking out a previous commit. Triggers are even worse - “spooky actions at a distance”.
    - zaphirplane 2y ago
      In your post you talk about forking and rolling back. I bet the same can be done for SP, even if is a fork and add suffix approach The calling code can call either On a phone hope I’m clear enough
      - scarface_74 2y ago
        If I need to fork code am I going to then make copies of all of the stored procedures corresponding with the code and now have getCustomer1, getCustomer2… and doing the same with all of your procedures? And then do I need to change all of my stored procedure references in my code? Do I merge my changes back to the main one when done? Isn’t just doing a “git branch” easier? Rolling back, is just doing a git revert in your separate Kubernetes repository (you are doing proper Gitops aren’t you?), recommitting your code and your cluster is now using your old Docker container. If you aren’t using K8s, you just push your old code.
csomar 2y ago
This throws away most of what your database can do for you. I personally like to do authorization at the database layer. There are a tons of other things a DB can do, like trigger a webhook. Are you going to frequently pull from your DB for live updates? I think OP issue is that he didn't understand what mock tests actually tested against. They are called mock for a reason. There should be a better way to provision a database when running tests and have a full simulated environment.
- Spivak 2y ago
  I think you've swung the other way a bit too far, I've never seen application logic split between the app and database not become a huge PITA. I would never have my DB trigger a webhook for me but I would have it act as a message broker back to my own code that triggers a webhook. I don't want my DB ever initiating an outbound connection.
- motorest 2y ago
  > This throws away most of what your database can do for you. I personally like to do authorization at the database layer. The whole point of writing maintainable software is to throw away most of what technologies can do, because you are way better off if you do not make the mistake of following that path. For example, at a first glance doing auth at the database layer is a huge mistake. You design your systems with a few nodes having exclusive access to the DB to ensure your access is secure without having to waste the overhead of checking credentials in each db access. The main design trait is speed. But you do you. > There are a tons of other things a DB can do, like trigger a webhook. There are plenty of reasons and hard learned lessons behind principles such as separation of concerns.
icedchai 2y ago
I haven't seen the "stored procs only"-style of DB development since the early 2000's, and I'd never go back to that.
- ramses0 2y ago
  Lambda functions would like to have a word...
  - icedchai 2y ago
    At least with Lambda you can use mainstream programming languages. You can also have single lambdas that handle multiple endpoints or events. It’s a bit different.
- SOLAR_FIELDS 2y ago
  For every “I made this way more efficient/clean with a stored procedure” there’s some incident where the stored procedure was left off, fired too many times, or just caused some problem due to its lack of visibility to the end user in the codebase
  - Two4 2y ago
    Exactly. I only break out SPs when I absolutely need to squeeze as much performance from my data store as possible. I'd rather pay for more horizontal scaling than push my logic across a system boundary where it's difficult to monitor and debug, because bugs in SPs are probably the single costliest class of bug in a prod system.
    - SOLAR_FIELDS 2y ago
      I’ve always felt this was partly database vendors fault for not having tooling that integrates well. I think there is some theoretical world where pgTAP and other database specific test harnesses work fairly seamlessly with language runtimes via some well designed SDK. Tracing would need to walk into the db call stack from the application code. Clearly this is harder to get right than expected since there aren’t many places doing stuff like that, that I’m aware of at least. Maybe if some of these database vendors built out better API’s for things like that with the expectation other language runtimes would be calling them the situation would be better. But it seems a lot of the old school db world is perfectly happy to build their products as if they were a black box from the application’s perspective
motorest 2y ago
> Don’t code with mocks period. Structure your code such that it has a functional core and imperative shell. All logic is unit testable (functional core). All IO and mutation Is not unit testable (imperative shell). I'm not sure what your definition of mocks is here. Even if you follow a principle where all IO and mutations is not unit testable, you still need test doubles to plug in the interface of your testable functional core. Because you need to pass data through to exercise code paths. How do you call the component that implements the interface of your dumb IO layer?
d0mine 2y ago
Here's one of the ways to test IO https://pypi.org/project/pytest-recording/ https://pypi.org/project/pytest-recording/ It can work very well in practice.

ljm 2y ago

so what you should do instead is write a load of pure functions and intersperse them with impure functions, and when it comes to testing that, if someone writes a different impure function than the one you wanted, you just fail the test and trigger a post to HN: "don't code with mocks."

ninetyninenine 2y ago
You failed to read and/or understand what I wrote. Also how do I fail a test for a new code addition I didn’t even write? Unit tests dont test impure functions all they do is test pure functions. Impure functions aren’t unit tested period. So when someone writes an impure function I don’t like what happens? Nothing. It wasn’t part of unit tests annyway, understand? If you’re referring to integration tests well a newly written impure function isn’t part of integration tests anyway so nothing happens in this case either. I found your last comment about how I’m triggered to write such a comment rude and lacking in even understanding what I wrote. Please try to understand my point before saying something like that… if you can’t please refrain from replying in such a way as it’s against the rules here. Thanks for reading.
- piva00 2y ago
  How do you write the expected return from an external API in your test setup?
  - ninetyninenine 2y ago
    That’s an integration test. Your external api returns a number, let’s say 10. You have two local functions. One that is very general and fetches that number, and another that transforms that number. Your local transformation function is the one that is tested. The other function is part of integration tests and in general can’t be unit tested.
    - piva00 2y ago
      Exactly, there's a layer (integration testing) where I usually do need to have a test against a mock to validate my function doing the external call is passing a request I expect. I can break it down into a parameter supplier function that will generate the proper request params and test that but I don't see any gain in abstracting that away simply to avoid mocks.
  - Spivak 2y ago
    You use a "VCR" module to record the real request and response and then play it back in your unit test. Boom you get the power of an integration test with the speed and determinism of a unit test. Then you turn off the VCR in CI to catch if/when the upstream API changes on you and now it's a real integration test. The problem with write tests, not too many, mostly integration is that unit tests are too damn good at giving you instant feedback and a tight dev loop. So we turned integration tests into unit tests. Combined with SQLite :memory: for unit tests and Postgres for the final integration tests we don't have to mock anything. Source: Currently use this method at $dayjob with great success.
    - piva00 2y ago
      My question was a rhetorical one, I do use request-replayers and other techniques (I've been through a bunch in the past 20+ years as they were developed). I was challenging the absolutism of "never use mocks", it's just another technique and can be applied easily in integration tests if the guidelines are well set in the team on how to not use them. Also, I do not do external calls in the CI/CD pipeline, it inevitably makes tests brittle and flaky.

throwawaymaths 2y ago

how do you test the imperative shell except with mocks? Suppose you're calling an external API

tomjakubowski 2y ago
The jargon around all this stuff is terrible. What I would probably write is, like, a fake version of that API, one which runs out of process from the library or test, and which can be "programmed" from the test set up -- "expect these requests, serve these responses", etc. Is that a mock? I don't know. It sure does resemble one, but it doesn't use dependency injection or other runtime tricks which I'd associate with classic "mocks"; it's just another program you write to support the tests. The imperative shell doing I/O needs to be configurable to support this (think a "widgets API base URL"), but that's it.
- ninetyninenine 2y ago
  The jargon is terrible because I think you misunderstood my point. You don’t mock period is my philosophy but your talking about mocking an entire api here.
- BoiledCabbage 2y ago
  Yes that still a mock. And thr point is to structure your code in such a way that there is little to no value left in testing that API call independently, because all of the related logic has been tested in a unit test. Once you've done that, you can just run a simple integration or end-to-end test to cover that API. You never need to manually mock it (like you suggested), or use a mocking tool.
  - drewcoo 2y ago
    > that still a mock It's a fake, not a mock. Mocks and fakes are both test doubles. > And thr point is to structure your code in such a way that Sometimes it's not your code. Sometimes you want to change legacy code and want tests to prove that your changes don't break anything. Life would be so much easier if there were just one way to test, wouldn't it?

paulryanrogers 2y ago

Please don't write a lot of stored procedures. Changing them in large or sharded DBs is not trivial and tooling to support them is minimal.

spawarotti 2y ago

Great advice. I follow it in my coding efforts and it has never failed me. Great book about this: Unit Testing Principles, Practices, and Patterns, Vladimir Khorikov, 2020 https://www.manning.com/books/unit-testing https://www.manning.com/books/unit-testing

dxxmxnd 2y ago

Any suggestions on where to read about this more?

tokonoma 2y ago
https://www.destroyallsoftware.com/talks/boundaries https://www.destroyallsoftware.com/talks/boundaries
- cryptonector 2y ago
  That's the only site whose dev contents I ever paid for. Gary Bernhardt is very entertaining.

bcrosby95 2y ago

> IO should be a dumb layer as much as possible. It should be Extremely general and as simple as fetching and requesting in the stupidest way possible. Then everything else should be covered by unit tests while IO is fundamentally not unit testable. Another way to put this is to ignore all the cool problems your database can solve for you and hand roll it instead. This also is a recipe for riddling your codebase with data races. You're trading fairly easy problems for very hard problems. No thanks. When running tests, optimize your database configuration for running tests against it, and make it possible to run those tests in parallel.

AndyNemmity 2y ago
Right, but you can see how they do it. The problems they are trading are likely not hard for them. Whatever you use, in your personal toolbox changes the results of the tradeoffs.

scarface_74 2y ago

I am actually at a point in my career that I won’t work for a company that is dependent on stored procedures. Rollbacks, versioning, branching, testing and everything else is a pain once you introduce stored procedures. As a lead for mostly green field applications over the past decade, my one no compromise architectural edict is no stored procedures for online production code. I might use it for background utilities or one off maintenance.

tbrownaw 2y ago
Is this about having application logic in multiple places, or having application logic co-located with the data? Does it make a difference if the procedures are database triggers vs being directly called by the application code? Would splitting off some of the logic into a separate service (that does its own database accesses) have the same issues?
- scarface_74 2y ago
  Say I have functionality to get customer data. If the sql logic is in code, I can create a new branch and work on it, merge my changes, I can do blue/green deployments. A rollback if you’re using Kubernetes is a simple matter of reverting back to the configuration file containing a reference to the old Docker container etc. How do you roll back a dozen stored procedures quickly? How do you just create a new branch or test a regression? With all of the logic in code, it’s a simple matter of checking out a previous commit. Triggers are even worse - “spooky actions at a distance”.
  - zaphirplane 2y ago
    In your post you talk about forking and rolling back. I bet the same can be done for SP, even if is a fork and add suffix approach The calling code can call either On a phone hope I’m clear enough
    - scarface_74 2y ago
      If I need to fork code am I going to then make copies of all of the stored procedures corresponding with the code and now have getCustomer1, getCustomer2… and doing the same with all of your procedures? And then do I need to change all of my stored procedure references in my code? Do I merge my changes back to the main one when done? Isn’t just doing a “git branch” easier? Rolling back, is just doing a git revert in your separate Kubernetes repository (you are doing proper Gitops aren’t you?), recommitting your code and your cluster is now using your old Docker container. If you aren’t using K8s, you just push your old code.

csomar 2y ago

This throws away most of what your database can do for you. I personally like to do authorization at the database layer. There are a tons of other things a DB can do, like trigger a webhook. Are you going to frequently pull from your DB for live updates? I think OP issue is that he didn't understand what mock tests actually tested against. They are called mock for a reason. There should be a better way to provision a database when running tests and have a full simulated environment.

Spivak 2y ago
I think you've swung the other way a bit too far, I've never seen application logic split between the app and database not become a huge PITA. I would never have my DB trigger a webhook for me but I would have it act as a message broker back to my own code that triggers a webhook. I don't want my DB ever initiating an outbound connection.
motorest 2y ago
> This throws away most of what your database can do for you. I personally like to do authorization at the database layer. The whole point of writing maintainable software is to throw away most of what technologies can do, because you are way better off if you do not make the mistake of following that path. For example, at a first glance doing auth at the database layer is a huge mistake. You design your systems with a few nodes having exclusive access to the DB to ensure your access is secure without having to waste the overhead of checking credentials in each db access. The main design trait is speed. But you do you. > There are a tons of other things a DB can do, like trigger a webhook. There are plenty of reasons and hard learned lessons behind principles such as separation of concerns.

icedchai 2y ago

I haven't seen the "stored procs only"-style of DB development since the early 2000's, and I'd never go back to that.

ramses0 2y ago
Lambda functions would like to have a word...
- icedchai 2y ago
  At least with Lambda you can use mainstream programming languages. You can also have single lambdas that handle multiple endpoints or events. It’s a bit different.
SOLAR_FIELDS 2y ago
For every “I made this way more efficient/clean with a stored procedure” there’s some incident where the stored procedure was left off, fired too many times, or just caused some problem due to its lack of visibility to the end user in the codebase
- Two4 2y ago
  Exactly. I only break out SPs when I absolutely need to squeeze as much performance from my data store as possible. I'd rather pay for more horizontal scaling than push my logic across a system boundary where it's difficult to monitor and debug, because bugs in SPs are probably the single costliest class of bug in a prod system.
  - SOLAR_FIELDS 2y ago
    I’ve always felt this was partly database vendors fault for not having tooling that integrates well. I think there is some theoretical world where pgTAP and other database specific test harnesses work fairly seamlessly with language runtimes via some well designed SDK. Tracing would need to walk into the db call stack from the application code. Clearly this is harder to get right than expected since there aren’t many places doing stuff like that, that I’m aware of at least. Maybe if some of these database vendors built out better API’s for things like that with the expectation other language runtimes would be calling them the situation would be better. But it seems a lot of the old school db world is perfectly happy to build their products as if they were a black box from the application’s perspective

motorest 2y ago

> Don’t code with mocks period. Structure your code such that it has a functional core and imperative shell. All logic is unit testable (functional core). All IO and mutation Is not unit testable (imperative shell). I'm not sure what your definition of mocks is here. Even if you follow a principle where all IO and mutations is not unit testable, you still need test doubles to plug in the interface of your testable functional core. Because you need to pass data through to exercise code paths. How do you call the component that implements the interface of your dumb IO layer?

d0mine 2y ago

Here's one of the ways to test IO https://pypi.org/project/pytest-recording/ https://pypi.org/project/pytest-recording/ It can work very well in practice.