5 ms·

For a person who does not code very large programs in Python, this looks scary and like something that is hardly "only one way to do it" and that the most elega

For a person who does not code very large programs in Python, this looks scary and like something that is hardly "only one way to do it" and that the most elegant solution gives you what you want. How I import things (or the libraries I depend on!) affects how I can write my tests? Really? My own Python scripts are typically single-screen in length. And one-off stuff, where they either work or don't, basically. Is this stuff really what developers of large-scale Python programs have to take into account? Or is this blog post misinformed, because there is an obviously better and standard way?

zzzeek 7y ago
no, this blog post is absolutely on point and correct. it can be extremely inconvenient and complicated to get around using mocks in many situations where you want to test things, so we want to use mocks when appropriate. Then, you definitely want to mock at the most specific level possible. while mocking in a way that is specific to how modules are imported is technically "fragile", in that it is deeply dependent on the structure code that's being tested, this is not an issue in practice, because the mocks are present in our test suites that run for every code change. If a code change moves around module assignments, our test will fail, and we know that we have to adjust the test to accommodate for the change. With appropriate continuous integration and code review practices, a broken mock-oriented test can't be inadvertently pushed into a repository. This does mean that when using mocks, you need to make sure mocks were called in the way that was expected for those cases where the code might silently move off using the mock in a way that wouldn't be detectable. The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get involved with actually creating filesystems which may be a complex and expensive process, especially if the test suite runs concurrent processes. if you mock the behavior of "os.listdir" to return a series of results, after the test code has been exercised, you usually want to assert using mock.mock_calls that the test code did in fact call the functions expected, unless it's clear in some other way that the test code definitely used that information. An example of this kind of code that I just helped someone with can be seen here: https://github.com/sqlalchemy/alembic/commit/02a1bf3454acb7b02942e246c19326630a8f9175#diff-c1de9c971ea0a629f46ecaf7dccad37dR817 https://github.com/sqlalchemy/alembic/commit/02a1bf3454acb7b... The Alembic test suite has a lot of test cases that go through all the trouble to build up real directory structures to test things, but that's a lot more work than just using a mock, so I use them where I can get away with this simpler approach.
- EdSchouten 7y ago
  > The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get involved with actually creating filesystems which may be a complex and expensive process, especially if the test suite runs concurrent processes. Alternative: have an abstract base class that describes a file system API. Have one implementation that builds on top of OS primitives and have another one that is a mock (potentially auto-generated through some mocking framework). That way there is no need to monkey-patch standard library functions at runtime. I did that within a project of mine, written in Go: https://github.com/buildbarn/bb-storage/blob/master/pkg/filesystem/directory.go https://github.com/buildbarn/bb-storage/blob/master/pkg/file... https://github.com/buildbarn/bb-storage/blob/master/pkg/filesystem/local_directory.go https://github.com/buildbarn/bb-storage/blob/master/pkg/file... Pretty slick that I can now use https://github.com/golang/mock https://github.com/golang/mock to automatically generate a mock of that.
  - zzzeek 7y ago
    Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. In this sense, mocks are solving the problem of having code that is full of dependency-injected AbstractFooBarFileSystemWithExtraPickles style of code, which is considered to be pretty un-Pythonic. I spent many years with Java and Spring so I can attest to both sides of this equation. Those of us using Python are using it because it is an interpreted, dynamic scripting language. If I'm coding in something more rigid like C or Go, then I'd expect to have a more complex architecture in order to achieve things that are fairly simple in a scripting language.
    - EdSchouten 7y ago
      > Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; It depends, right? If you suddenly wanted to let your existing set of classes read their inputs not from disk, but from some other kind of storage (e.g., files embedded in a Zip file), you'd only need to write one extra class and you're good to go. That would be a lot harder if your code called os.* directly. > a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. If by dependency injection system you mean invoking one extra constructor in, say, main() and pass the object along as a handle, sure.
wdroz 7y ago
In the alternative approaches cited by the author, the article "Itamar Turner-Trauring’s article" [0] presents an example that is more representative (IMO) on what people do for "mocking". [0] - https://pythonspeed.com/articles/verified-fakes/ https://pythonspeed.com/articles/verified-fakes/
coldtea 7y ago
>My own Python scripts are typically single-screen in length. And one-off stuff, where they either work or don't, basically. (...) Is this stuff really what developers of large-scale Python programs have to take into account? Not the only way to go about it, but yes, the underlying concerns are absolutely stuff that developers of large-scale Python programs have to take into account. At "single screen" length you can do anything you want. Really, anything goes -- and you could ever just stare at the code a lot to find all or almost all bugs. At 1000s of lines or more (can get into 100s or 1M lines), and especially on growing and worked on code (that gets new features, refactors, etc) you need to follow other practices to ensure it all works.
scbrg 7y ago
> How I import things (or the libraries I depend on!) affects how I can write my tests? Really? Well, no. Everything that needs to be understood is explained in the first section, A quick aside about assignment. Everything else is just discussion about logical consequences of this behavior. If you do an assignment (and "from foo import bar" is an assignment) you're just creating another name for a value (in this case, you're creating a name foo, in your namespace (presumably, but not necessarily a module) that points at whatever the name foo in module bar happened to point at at the moment you created it). If the original (or should I say, one of the previous) name(s) to the value is reassigned that does not affect what your name is pointing at. This is really all there is to know, and once you've grokked that, everything Python falls in place.

zzzeek 7y ago

no, this blog post is absolutely on point and correct. it can be extremely inconvenient and complicated to get around using mocks in many situations where you want to test things, so we want to use mocks when appropriate. Then, you definitely want to mock at the most specific level possible. while mocking in a way that is specific to how modules are imported is technically "fragile", in that it is deeply dependent on the structure code that's being tested, this is not an issue in practice, because the mocks are present in our test suites that run for every code change. If a code change moves around module assignments, our test will fail, and we know that we have to adjust the test to accommodate for the change. With appropriate continuous integration and code review practices, a broken mock-oriented test can't be inadvertently pushed into a repository. This does mean that when using mocks, you need to make sure mocks were called in the way that was expected for those cases where the code might silently move off using the mock in a way that wouldn't be detectable. The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get involved with actually creating filesystems which may be a complex and expensive process, especially if the test suite runs concurrent processes. if you mock the behavior of "os.listdir" to return a series of results, after the test code has been exercised, you usually want to assert using mock.mock_calls that the test code did in fact call the functions expected, unless it's clear in some other way that the test code definitely used that information. An example of this kind of code that I just helped someone with can be seen here: https://github.com/sqlalchemy/alembic/commit/02a1bf3454acb7b02942e246c19326630a8f9175#diff-c1de9c971ea0a629f46ecaf7dccad37dR817 https://github.com/sqlalchemy/alembic/commit/02a1bf3454acb7b... The Alembic test suite has a lot of test cases that go through all the trouble to build up real directory structures to test things, but that's a lot more work than just using a mock, so I use them where I can get away with this simpler approach.

EdSchouten 7y ago
> The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get involved with actually creating filesystems which may be a complex and expensive process, especially if the test suite runs concurrent processes. Alternative: have an abstract base class that describes a file system API. Have one implementation that builds on top of OS primitives and have another one that is a mock (potentially auto-generated through some mocking framework). That way there is no need to monkey-patch standard library functions at runtime. I did that within a project of mine, written in Go: https://github.com/buildbarn/bb-storage/blob/master/pkg/filesystem/directory.go https://github.com/buildbarn/bb-storage/blob/master/pkg/file... https://github.com/buildbarn/bb-storage/blob/master/pkg/filesystem/local_directory.go https://github.com/buildbarn/bb-storage/blob/master/pkg/file... Pretty slick that I can now use https://github.com/golang/mock https://github.com/golang/mock to automatically generate a mock of that.
- zzzeek 7y ago
  Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. In this sense, mocks are solving the problem of having code that is full of dependency-injected AbstractFooBarFileSystemWithExtraPickles style of code, which is considered to be pretty un-Pythonic. I spent many years with Java and Spring so I can attest to both sides of this equation. Those of us using Python are using it because it is an interpreted, dynamic scripting language. If I'm coding in something more rigid like C or Go, then I'd expect to have a more complex architecture in order to achieve things that are fairly simple in a scripting language.
  - EdSchouten 7y ago
    > Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; It depends, right? If you suddenly wanted to let your existing set of classes read their inputs not from disk, but from some other kind of storage (e.g., files embedded in a Zip file), you'd only need to write one extra class and you're good to go. That would be a lot harder if your code called os.* directly. > a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. If by dependency injection system you mean invoking one extra constructor in, say, main() and pass the object along as a handle, sure.

wdroz 7y ago

In the alternative approaches cited by the author, the article "Itamar Turner-Trauring’s article" [0] presents an example that is more representative (IMO) on what people do for "mocking". [0] - https://pythonspeed.com/articles/verified-fakes/ https://pythonspeed.com/articles/verified-fakes/

coldtea 7y ago

>My own Python scripts are typically single-screen in length. And one-off stuff, where they either work or don't, basically. (...) Is this stuff really what developers of large-scale Python programs have to take into account? Not the only way to go about it, but yes, the underlying concerns are absolutely stuff that developers of large-scale Python programs have to take into account. At "single screen" length you can do anything you want. Really, anything goes -- and you could ever just stare at the code a lot to find all or almost all bugs. At 1000s of lines or more (can get into 100s or 1M lines), and especially on growing and worked on code (that gets new features, refactors, etc) you need to follow other practices to ensure it all works.

scbrg 7y ago

> How I import things (or the libraries I depend on!) affects how I can write my tests? Really? Well, no. Everything that needs to be understood is explained in the first section, A quick aside about assignment. Everything else is just discussion about logical consequences of this behavior. If you do an assignment (and "from foo import bar" is an assignment) you're just creating another name for a value (in this case, you're creating a name foo, in your namespace (presumably, but not necessarily a module) that points at whatever the name foo in module bar happened to point at at the moment you created it). If the original (or should I say, one of the previous) name(s) to the value is reassigned that does not affect what your name is pointing at. This is really all there is to know, and once you've grokked that, everything Python falls in place.