5 ms·

This blog post talks as if mocking the `open` function is a good thing that people should be told how to do. If you are mocking anything in the standard library

by dherls 11mo ago

This blog post talks as if mocking the `open` function is a good thing that people should be told how to do. If you are mocking anything in the standard library your code is probably structured poorly. In the example the author walks through, a cleaner way would be to have the second function take the Options as a parameter and decouple those two functions. You can then test both in isolation.

bluGill 11mo ago
Details matters, but good test doubles here are important. You want to capture all calls to IO and do something different. You don't want tests to break because someone has a different filesystem, didn't set their home directory as you want it setup, or worse is trying to run two different tests at the same time and the other test is changing files the other wants. Note that I said test doubles. Mocks are a bit over specific - they are about verifying functions are called at the right time with the right arguments, but the easy ability to set return values makes it easy to abuse them for other things (this abuse is good, but it is still abuse of the intent). In this case you want a fake: a smart service that when you are in a test setups a temporary directory tree that contains all the files you need in the state that particular test needs, and destroys that when the test is done (with an optional mode to keep it - useful if a test fails to see debug). Depending on your situation you may need something for network services, time, or other such things. Note that in most cases a filesystem itself is more than fast enough to use in tests, but you need isolation from other tests. There are a number of ways to create this fake, it could override open, or it could just be a GetMyProgramDir function that you override are two that I can think of.
- jpollock 11mo ago
  Your tests are either hermetic, or they're flaky. That means the test environment needs to be defined and versioned with the code.
- dherls 11mo ago
  Even in the case you mention you really shouldn't be overriding these methods. Your load settings method should take the path of the settings file as an argument, and then your test can set up all the fake files you want with something like python's tempfile package
  - bluGill 11mo ago
    There are a number of different ways to solve this problem. I too use the path of settings in my code, but I'm not against overriding open and all the other file io functions. Of course this article is about python which has different abilities than other languages, what is best in python is not what is best in other languages, and I'm trying to stay at a higher level that a particular language.
1718627440 11mo ago
> This blog post talks as if mocking the `open` function is a good thing that people should be told how to do. If you are mocking anything in the standard library your code is probably structured poorly. Valgrind is a mock of standard library/OS functions and I think its existence is a good thing. Simulating OOM is also only possible by mocking stuff like open.
- vkou 11mo ago
  All rules exist to be broken in the right circumstances. But in 99.9% of test code, there's no reason to do any of that.
  - 1718627440 11mo ago
    I think when testing code with an open call, it is a good idea to test what happens on different return values of open. If that is not what you intent to test for this method, then that method shouldn't contain open at all, as already pointed out by other comments.
    - vkou 11mo ago
      That depends on what your error recovery plan is. If the code's running in a space shuttle, you probably want to test that path. If it's bootstrapping a replicated service, it's likely desirable to crash early if a config file couldn't be opened. If it's plausible that the file in question is missing, you can absolutely test that code path, without mocking open. If you want to explicitly handle different reasons for why opening a file failed differently, by all means, stress all of that in your tests. But if all you have is a happy path and an unhappy path, where your code doesn't care why opening a file failed, all you need to test is the case where the file is present, and one where it is not.
      - 1718627440 11mo ago
        Modifying the file system to be would be kind of like mocking to me. I very much, don't want my daemons or user-facing applications to just crash, when a file is missing. That's kind-of the worst thing you can do.
sunrunner 11mo ago
> If you are mocking anything in the standard library your code is probably structured poorly. I like Hynek Schlawak's 'Don’t Mock What You Don’t Own' [1] phrasing, and while I'm not a fan of adding too many layers of abstraction to an application that hasn't proved that it needs them, the one structure I find consistently useful is to add a very thin layer over parts that do I/O, converting to/from types that you own to whatever is needed for the actual thing. These layers should be boring and narrow (for example, never mock past validation you depend upon), doing as little conversion as possible. You can also rephrase the general purpose open()-type usage into application/purpose-specific usages of that. Then you can either unittest.mock.patch these or provide alternate stub implementations for tests in a different way, with this this approach also translating easily to other languages that don't have the (double-edged sword) flexibility of Python's own unittest.mock. [1] https://hynek.me/articles/what-to-mock-in-5-mins/ https://hynek.me/articles/what-to-mock-in-5-mins/
vkou 11mo ago
> This blog post talks as if mocking the `open` function is a good thing that people should be told how to do. It does. And this is exactly the problem, here! > TFA: The thing we want to avoid is opening a real file No! No, no, no! You do not 'want to avoid opening a real file' in a test. It's completely fine to open a real file in a test! If your code depends on reading input files, then your test should include real input files in it! There's no reason to mock any of this. All of this stuff is easy to set up in any unit test library worth it's salt.
- 9rx 11mo ago
  > then your test should include real input files in it! There's no reason to mock any of this. That's okay for testing some branches of your code. But not all. I don't want to have to actually crash my hard drive to test that I am properly handling hard drive crashes. Mocking[1] is the easiest way to do that. [1] For some definition of mock. There is absolutely no agreement found in this space as to what the terms used mean.

by dherls 11mo ago

bluGill 11mo ago

Details matters, but good test doubles here are important. You want to capture all calls to IO and do something different. You don't want tests to break because someone has a different filesystem, didn't set their home directory as you want it setup, or worse is trying to run two different tests at the same time and the other test is changing files the other wants. Note that I said test doubles. Mocks are a bit over specific - they are about verifying functions are called at the right time with the right arguments, but the easy ability to set return values makes it easy to abuse them for other things (this abuse is good, but it is still abuse of the intent). In this case you want a fake: a smart service that when you are in a test setups a temporary directory tree that contains all the files you need in the state that particular test needs, and destroys that when the test is done (with an optional mode to keep it - useful if a test fails to see debug). Depending on your situation you may need something for network services, time, or other such things. Note that in most cases a filesystem itself is more than fast enough to use in tests, but you need isolation from other tests. There are a number of ways to create this fake, it could override open, or it could just be a GetMyProgramDir function that you override are two that I can think of.

jpollock 11mo ago
Your tests are either hermetic, or they're flaky. That means the test environment needs to be defined and versioned with the code.
dherls 11mo ago
Even in the case you mention you really shouldn't be overriding these methods. Your load settings method should take the path of the settings file as an argument, and then your test can set up all the fake files you want with something like python's tempfile package
- bluGill 11mo ago
  There are a number of different ways to solve this problem. I too use the path of settings in my code, but I'm not against overriding open and all the other file io functions. Of course this article is about python which has different abilities than other languages, what is best in python is not what is best in other languages, and I'm trying to stay at a higher level that a particular language.

1718627440 11mo ago

> This blog post talks as if mocking the `open` function is a good thing that people should be told how to do. If you are mocking anything in the standard library your code is probably structured poorly. Valgrind is a mock of standard library/OS functions and I think its existence is a good thing. Simulating OOM is also only possible by mocking stuff like open.

vkou 11mo ago
All rules exist to be broken in the right circumstances. But in 99.9% of test code, there's no reason to do any of that.
- 1718627440 11mo ago
  I think when testing code with an open call, it is a good idea to test what happens on different return values of open. If that is not what you intent to test for this method, then that method shouldn't contain open at all, as already pointed out by other comments.
  - vkou 11mo ago
    That depends on what your error recovery plan is. If the code's running in a space shuttle, you probably want to test that path. If it's bootstrapping a replicated service, it's likely desirable to crash early if a config file couldn't be opened. If it's plausible that the file in question is missing, you can absolutely test that code path, without mocking open. If you want to explicitly handle different reasons for why opening a file failed differently, by all means, stress all of that in your tests. But if all you have is a happy path and an unhappy path, where your code doesn't care why opening a file failed, all you need to test is the case where the file is present, and one where it is not.
    - 1718627440 11mo ago
      Modifying the file system to be would be kind of like mocking to me. I very much, don't want my daemons or user-facing applications to just crash, when a file is missing. That's kind-of the worst thing you can do.

sunrunner 11mo ago

> If you are mocking anything in the standard library your code is probably structured poorly. I like Hynek Schlawak's 'Don’t Mock What You Don’t Own' [1] phrasing, and while I'm not a fan of adding too many layers of abstraction to an application that hasn't proved that it needs them, the one structure I find consistently useful is to add a very thin layer over parts that do I/O, converting to/from types that you own to whatever is needed for the actual thing. These layers should be boring and narrow (for example, never mock past validation you depend upon), doing as little conversion as possible. You can also rephrase the general purpose open()-type usage into application/purpose-specific usages of that. Then you can either unittest.mock.patch these or provide alternate stub implementations for tests in a different way, with this this approach also translating easily to other languages that don't have the (double-edged sword) flexibility of Python's own unittest.mock. [1] https://hynek.me/articles/what-to-mock-in-5-mins/ https://hynek.me/articles/what-to-mock-in-5-mins/

vkou 11mo ago

> This blog post talks as if mocking the `open` function is a good thing that people should be told how to do. It does. And this is exactly the problem, here! > TFA: The thing we want to avoid is opening a real file No! No, no, no! You do not 'want to avoid opening a real file' in a test. It's completely fine to open a real file in a test! If your code depends on reading input files, then your test should include real input files in it! There's no reason to mock any of this. All of this stuff is easy to set up in any unit test library worth it's salt.

9rx 11mo ago
> then your test should include real input files in it! There's no reason to mock any of this. That's okay for testing some branches of your code. But not all. I don't want to have to actually crash my hard drive to test that I am properly handling hard drive crashes. Mocking[1] is the easiest way to do that. [1] For some definition of mock. There is absolutely no agreement found in this space as to what the terms used mean.