4 ms·

no, this blog post is absolutely on point and correct. it can be extremely inconvenient and complicated to get around using mocks in many situations where you

by zzzeek 7y ago

no, this blog post is absolutely on point and correct. it can be extremely inconvenient and complicated to get around using mocks in many situations where you want to test things, so we want to use mocks when appropriate. Then, you definitely want to mock at the most specific level possible. while mocking in a way that is specific to how modules are imported is technically "fragile", in that it is deeply dependent on the structure code that's being tested, this is not an issue in practice, because the mocks are present in our test suites that run for every code change. If a code change moves around module assignments, our test will fail, and we know that we have to adjust the test to accommodate for the change. With appropriate continuous integration and code review practices, a broken mock-oriented test can't be inadvertently pushed into a repository. This does mean that when using mocks, you need to make sure mocks were called in the way that was expected for those cases where the code might silently move off using the mock in a way that wouldn't be detectable. The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get involved with actually creating filesystems which may be a complex and expensive process, especially if the test suite runs concurrent processes. if you mock the behavior of "os.listdir" to return a series of results, after the test code has been exercised, you usually want to assert using mock.mock_calls that the test code did in fact call the functions expected, unless it's clear in some other way that the test code definitely used that information. An example of this kind of code that I just helped someone with can be seen here: https://github.com/sqlalchemy/alembic/commit/02a1bf3454acb7b02942e246c19326630a8f9175#diff-c1de9c971ea0a629f46ecaf7dccad37dR817 https://github.com/sqlalchemy/alembic/commit/02a1bf3454acb7b... The Alembic test suite has a lot of test cases that go through all the trouble to build up real directory structures to test things, but that's a lot more work than just using a mock, so I use them where I can get away with this simpler approach.

EdSchouten 7y ago
> The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get involved with actually creating filesystems which may be a complex and expensive process, especially if the test suite runs concurrent processes. Alternative: have an abstract base class that describes a file system API. Have one implementation that builds on top of OS primitives and have another one that is a mock (potentially auto-generated through some mocking framework). That way there is no need to monkey-patch standard library functions at runtime. I did that within a project of mine, written in Go: https://github.com/buildbarn/bb-storage/blob/master/pkg/filesystem/directory.go https://github.com/buildbarn/bb-storage/blob/master/pkg/file... https://github.com/buildbarn/bb-storage/blob/master/pkg/filesystem/local_directory.go https://github.com/buildbarn/bb-storage/blob/master/pkg/file... Pretty slick that I can now use https://github.com/golang/mock https://github.com/golang/mock to automatically generate a mock of that.
- zzzeek 7y ago
  Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. In this sense, mocks are solving the problem of having code that is full of dependency-injected AbstractFooBarFileSystemWithExtraPickles style of code, which is considered to be pretty un-Pythonic. I spent many years with Java and Spring so I can attest to both sides of this equation. Those of us using Python are using it because it is an interpreted, dynamic scripting language. If I'm coding in something more rigid like C or Go, then I'd expect to have a more complex architecture in order to achieve things that are fairly simple in a scripting language.
  - EdSchouten 7y ago
    > Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; It depends, right? If you suddenly wanted to let your existing set of classes read their inputs not from disk, but from some other kind of storage (e.g., files embedded in a Zip file), you'd only need to write one extra class and you're good to go. That would be a lot harder if your code called os.* directly. > a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. If by dependency injection system you mean invoking one extra constructor in, say, main() and pass the object along as a handle, sure.
    - zzzeek 7y ago
      well again, I grew up on GOF programming and once I grokked how mocks in Python worked, I was very glad to embrace their approach, which has allowed me to write much simpler code that is more thoroughly tested; I of course still use abstractions to a great degree, but I no longer have to build out an abstraction system when I just want to make sure some fairly straightforward code is fully tested. I no longer have to build out everything as an abstraction when such a system is generally YAGNI, I can use the Python standard library directly. This allows me to do less work, write and maintain less code, and have better test coverage. It allows my code to be more fully tested even when it has not yet been abstracted, if that's what's in store for it. Patching local imports within the scope of two lines of code is a non-issue thanks to Python context managers. I have much more complicated examples of code that was already plenty complicated and mocks allowed me to get it tested quickly and effectively, instead of having to break it out into even more complexity. Basically mocks have been all productivity and no downside for me whatsoever, using the Python standard library mock which is extremely well designed.
kdazzle 7y ago
Yes - totally agree. Sure, mocking everything is a bad idea. And mocking can make changes brittle. But I would much rather just mock HTTP requests, dates, and filesystem stuff than spend a day figuring out how to write some wrapper around HTTP requests to inject in .NET core.
adrianmonk 7y ago
I think they are asking a question about Python, not about mocks. I was also very surprised Python does it this way. Normally I would expect "import" statements in a language to just rearrange the namespace and have no other effects at all. The author explains that it doesn't work that way: > “from mod import val” means, import mod, and then do the assignment “val = mod.val”. In other words, I would expect an import to alter a symbol table. I would not expect it to create a new variable and then perform an assignment.
- nedbat 7y ago
  How is "altering a symbol table" different than "creating a new variable"? Imports have to create a new variable of some sort: their entire point is to provide a name that you can use. "import mod" defines mod, which didn't exist in your module before. "from mod import val" defines val, which didn't exist in your module before.
  - zzzeek 7y ago
    they likely are referring to how import mechanisms in other languages operate outside the scope of imperative execution, like Perl's "use" statement, or in the way that a compiled language like Java handles imports. It is exactly the vast confusion that Perl's "use" caused me, even after I used Perl in a professional setting for almost ten years, that allowed Python's "first class object" approach to imports to be one of the most liberating breaths of fresh air I've ever had in my programming career. Of course, imports being imperatively executed causes the nasty problem of import dependency cycles in code but it doesn't even bother me.
    - adrianmonk 7y ago
      It's not specific to compiled languages. An interpreter has access to the symbol table while interpreting your code in the same way that a compiler has access to it while compiling it.
  - adrianmonk 7y ago
    For a moment, let's talk in general terms, not about how Python chose to do it. Most languages have scopes. Things may exist in your scope, so you can access them. Or they may exist but in some other scope not visible to you where you currently are. Symbol tables track what is in a scope. A variable is a name associated with a storage location. The location could be a stack, heap, register, or something way more abstract as long as it behaves as a "place" you can store a value into and read from. When a variable is created, a name is defined and some storage is arranged for. But that name is also added to a scope. (If it weren't, it would have a name, but nobody could see it.) There's no reason, once the variable has been added to one scope, that it can't be added to another scope. That is what "import" statements do in a lot of languages. Python has apparently chosen a very different approach for imports, which is to create a new variable. This isn't necessarily wrong or right, it's just not something I had ever seen a language do before.

by zzzeek 7y ago

EdSchouten 7y ago

> The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get involved with actually creating filesystems which may be a complex and expensive process, especially if the test suite runs concurrent processes. Alternative: have an abstract base class that describes a file system API. Have one implementation that builds on top of OS primitives and have another one that is a mock (potentially auto-generated through some mocking framework). That way there is no need to monkey-patch standard library functions at runtime. I did that within a project of mine, written in Go: https://github.com/buildbarn/bb-storage/blob/master/pkg/filesystem/directory.go https://github.com/buildbarn/bb-storage/blob/master/pkg/file... https://github.com/buildbarn/bb-storage/blob/master/pkg/filesystem/local_directory.go https://github.com/buildbarn/bb-storage/blob/master/pkg/file... Pretty slick that I can now use https://github.com/golang/mock https://github.com/golang/mock to automatically generate a mock of that.

zzzeek 7y ago
Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. In this sense, mocks are solving the problem of having code that is full of dependency-injected AbstractFooBarFileSystemWithExtraPickles style of code, which is considered to be pretty un-Pythonic. I spent many years with Java and Spring so I can attest to both sides of this equation. Those of us using Python are using it because it is an interpreted, dynamic scripting language. If I'm coding in something more rigid like C or Go, then I'd expect to have a more complex architecture in order to achieve things that are fairly simple in a scripting language.
- EdSchouten 7y ago
  > Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; It depends, right? If you suddenly wanted to let your existing set of classes read their inputs not from disk, but from some other kind of storage (e.g., files embedded in a Zip file), you'd only need to write one extra class and you're good to go. That would be a lot harder if your code called os.* directly. > a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. If by dependency injection system you mean invoking one extra constructor in, say, main() and pass the object along as a handle, sure.
  - zzzeek 7y ago
    well again, I grew up on GOF programming and once I grokked how mocks in Python worked, I was very glad to embrace their approach, which has allowed me to write much simpler code that is more thoroughly tested; I of course still use abstractions to a great degree, but I no longer have to build out an abstraction system when I just want to make sure some fairly straightforward code is fully tested. I no longer have to build out everything as an abstraction when such a system is generally YAGNI, I can use the Python standard library directly. This allows me to do less work, write and maintain less code, and have better test coverage. It allows my code to be more fully tested even when it has not yet been abstracted, if that's what's in store for it. Patching local imports within the scope of two lines of code is a non-issue thanks to Python context managers. I have much more complicated examples of code that was already plenty complicated and mocks allowed me to get it tested quickly and effectively, instead of having to break it out into even more complexity. Basically mocks have been all productivity and no downside for me whatsoever, using the Python standard library mock which is extremely well designed.

kdazzle 7y ago

Yes - totally agree. Sure, mocking everything is a bad idea. And mocking can make changes brittle. But I would much rather just mock HTTP requests, dates, and filesystem stuff than spend a day figuring out how to write some wrapper around HTTP requests to inject in .NET core.

adrianmonk 7y ago

I think they are asking a question about Python, not about mocks. I was also very surprised Python does it this way. Normally I would expect "import" statements in a language to just rearrange the namespace and have no other effects at all. The author explains that it doesn't work that way: > “from mod import val” means, import mod, and then do the assignment “val = mod.val”. In other words, I would expect an import to alter a symbol table. I would not expect it to create a new variable and then perform an assignment.

nedbat 7y ago
How is "altering a symbol table" different than "creating a new variable"? Imports have to create a new variable of some sort: their entire point is to provide a name that you can use. "import mod" defines mod, which didn't exist in your module before. "from mod import val" defines val, which didn't exist in your module before.
- zzzeek 7y ago
  they likely are referring to how import mechanisms in other languages operate outside the scope of imperative execution, like Perl's "use" statement, or in the way that a compiled language like Java handles imports. It is exactly the vast confusion that Perl's "use" caused me, even after I used Perl in a professional setting for almost ten years, that allowed Python's "first class object" approach to imports to be one of the most liberating breaths of fresh air I've ever had in my programming career. Of course, imports being imperatively executed causes the nasty problem of import dependency cycles in code but it doesn't even bother me.
  - adrianmonk 7y ago
    It's not specific to compiled languages. An interpreter has access to the symbol table while interpreting your code in the same way that a compiler has access to it while compiling it.
- adrianmonk 7y ago
  For a moment, let's talk in general terms, not about how Python chose to do it. Most languages have scopes. Things may exist in your scope, so you can access them. Or they may exist but in some other scope not visible to you where you currently are. Symbol tables track what is in a scope. A variable is a name associated with a storage location. The location could be a stack, heap, register, or something way more abstract as long as it behaves as a "place" you can store a value into and read from. When a variable is created, a name is defined and some storage is arranged for. But that name is also added to a scope. (If it weren't, it would have a name, but nobody could see it.) There's no reason, once the variable has been added to one scope, that it can't be added to another scope. That is what "import" statements do in a lot of languages. Python has apparently chosen a very different approach for imports, which is to create a new variable. This isn't necessarily wrong or right, it's just not something I had ever seen a language do before.