6 ms·

> The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get inv

> The "os.listdir()" example in this blog is a pretty common case, using mocks to test code that works with filesystems, where you don't need or want to get involved with actually creating filesystems which may be a complex and expensive process, especially if the test suite runs concurrent processes. Alternative: have an abstract base class that describes a file system API. Have one implementation that builds on top of OS primitives and have another one that is a mock (potentially auto-generated through some mocking framework). That way there is no need to monkey-patch standard library functions at runtime. I did that within a project of mine, written in Go: https://github.com/buildbarn/bb-storage/blob/master/pkg/filesystem/directory.go https://github.com/buildbarn/bb-storage/blob/master/pkg/file... https://github.com/buildbarn/bb-storage/blob/master/pkg/filesystem/local_directory.go https://github.com/buildbarn/bb-storage/blob/master/pkg/file... Pretty slick that I can now use https://github.com/golang/mock https://github.com/golang/mock to automatically generate a mock of that.

zzzeek 7y ago
Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. In this sense, mocks are solving the problem of having code that is full of dependency-injected AbstractFooBarFileSystemWithExtraPickles style of code, which is considered to be pretty un-Pythonic. I spent many years with Java and Spring so I can attest to both sides of this equation. Those of us using Python are using it because it is an interpreted, dynamic scripting language. If I'm coding in something more rigid like C or Go, then I'd expect to have a more complex architecture in order to achieve things that are fairly simple in a scripting language.
- EdSchouten 7y ago
  > Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; It depends, right? If you suddenly wanted to let your existing set of classes read their inputs not from disk, but from some other kind of storage (e.g., files embedded in a Zip file), you'd only need to write one extra class and you're good to go. That would be a lot harder if your code called os.* directly. > a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. If by dependency injection system you mean invoking one extra constructor in, say, main() and pass the object along as a handle, sure.
  - zzzeek 7y ago
    well again, I grew up on GOF programming and once I grokked how mocks in Python worked, I was very glad to embrace their approach, which has allowed me to write much simpler code that is more thoroughly tested; I of course still use abstractions to a great degree, but I no longer have to build out an abstraction system when I just want to make sure some fairly straightforward code is fully tested. I no longer have to build out everything as an abstraction when such a system is generally YAGNI, I can use the Python standard library directly. This allows me to do less work, write and maintain less code, and have better test coverage. It allows my code to be more fully tested even when it has not yet been abstracted, if that's what's in store for it. Patching local imports within the scope of two lines of code is a non-issue thanks to Python context managers. I have much more complicated examples of code that was already plenty complicated and mocks allowed me to get it tested quickly and effectively, instead of having to break it out into even more complexity. Basically mocks have been all productivity and no downside for me whatsoever, using the Python standard library mock which is extremely well designed.
- coldtea 7y ago
  >Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain Or easier - as the abstraction can simplify the interface, localize the logic for directory work, make it easier to port, and so on. That's the reason why most of us use "requests" and not urllib, too, for example, or Yoda and not Java's legacy time mess. >a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. You can write alterative implementations (whether with base class, interface, traits, or what have you) and use them for production code and for testing without any dependency injection. >Those of us using Python are using it because it is an interpreted, dynamic scripting language. The abstractions mentioned come from Smalltalk, which arguably is the same or even more dynamic than Python. Having a class/interface/protocol/trait and a test implementation is by no means a static typing/Java thing, or counter to a dynamic language, as implied here...
- nfrbc 7y ago
  > a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. Dependency injection can be as simple as: class Printer: def run(self): print("hello world") class Caller: def __init__(self, printer_class=Printer): self.printer = printer_class() def doit(self): self.printer.run() In your tests all you have to do is pass a different object to your Caller class. No need to mess with factories, interfaces or Impl suffixes. If the point is just having simpler testability and making dependencies more explicit, this gets the job done. And since Python is a dynamic language, you can use this with stdlib classes too, so it solves your first complaint. I've seen it a lot in Ruby and C# lately.
SomeOldThrow 7y ago
It’s not always possible or pragmatic to entirely rewrite the consumption of an interface just to allow dependency injection.

zzzeek 7y ago

Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. In this sense, mocks are solving the problem of having code that is full of dependency-injected AbstractFooBarFileSystemWithExtraPickles style of code, which is considered to be pretty un-Pythonic. I spent many years with Java and Spring so I can attest to both sides of this equation. Those of us using Python are using it because it is an interpreted, dynamic scripting language. If I'm coding in something more rigid like C or Go, then I'd expect to have a more complex architecture in order to achieve things that are fairly simple in a scripting language.

EdSchouten 7y ago
> Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain; It depends, right? If you suddenly wanted to let your existing set of classes read their inputs not from disk, but from some other kind of storage (e.g., files embedded in a Zip file), you'd only need to write one extra class and you're good to go. That would be a lot harder if your code called os.* directly. > a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. If by dependency injection system you mean invoking one extra constructor in, say, main() and pass the object along as a handle, sure.
- zzzeek 7y ago
  well again, I grew up on GOF programming and once I grokked how mocks in Python worked, I was very glad to embrace their approach, which has allowed me to write much simpler code that is more thoroughly tested; I of course still use abstractions to a great degree, but I no longer have to build out an abstraction system when I just want to make sure some fairly straightforward code is fully tested. I no longer have to build out everything as an abstraction when such a system is generally YAGNI, I can use the Python standard library directly. This allows me to do less work, write and maintain less code, and have better test coverage. It allows my code to be more fully tested even when it has not yet been abstracted, if that's what's in store for it. Patching local imports within the scope of two lines of code is a non-issue thanks to Python context managers. I have much more complicated examples of code that was already plenty complicated and mocks allowed me to get it tested quickly and effectively, instead of having to break it out into even more complexity. Basically mocks have been all productivity and no downside for me whatsoever, using the Python standard library mock which is extremely well designed.
coldtea 7y ago
>Sure but then I have to write my real library code using an abstraction, making my code more difficult to read and maintain Or easier - as the abstraction can simplify the interface, localize the logic for directory work, make it easier to port, and so on. That's the reason why most of us use "requests" and not urllib, too, for example, or Yoda and not Java's legacy time mess. >a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. You can write alterative implementations (whether with base class, interface, traits, or what have you) and use them for production code and for testing without any dependency injection. >Those of us using Python are using it because it is an interpreted, dynamic scripting language. The abstractions mentioned come from Smalltalk, which arguably is the same or even more dynamic than Python. Having a class/interface/protocol/trait and a test implementation is by no means a static typing/Java thing, or counter to a dynamic language, as implied here...
nfrbc 7y ago
> a dependency injection system is then necessary in order to have the correct concrete implementations set up at runtime. Dependency injection can be as simple as: class Printer: def run(self): print("hello world") class Caller: def __init__(self, printer_class=Printer): self.printer = printer_class() def doit(self): self.printer.run() In your tests all you have to do is pass a different object to your Caller class. No need to mess with factories, interfaces or Impl suffixes. If the point is just having simpler testability and making dependencies more explicit, this gets the job done. And since Python is a dynamic language, you can use this with stdlib classes too, so it solves your first complaint. I've seen it a lot in Ruby and C# lately.

SomeOldThrow 7y ago

It’s not always possible or pragmatic to entirely rewrite the consumption of an interface just to allow dependency injection.