7 ms·
Minimalist dependency injection in Python (2021)
- pansa2 5y ago> Minimalist dependency injection in Python When an object requires other objects for its work, pass references to those other objects into its __init__ method. That’s it.
- rmbyrro 5y agoYou clearly haven't read the article
- programmarchy 5y agoYo dawg, I heard you like dependencies, so we created a dependency injector dependency for you to inject so you can inject your dependencies. Sorry couldn't resist. The additional dependency is the biggest problem I have with OP's approach though. Now none of my classes are easily portable across projects, and this DI framework is going to spread around everywhere.
- ldlework 5y agoAll you've done is push the burden of constructing the dependency to whomever is calling an passing that dependency to __init__ Consistently applied, all construction gets pushed to the entry-point of the program. Congratulations, you've just discovered the so-called "composition root". Now that all construction is taking place at once, the order matters as you can't pass a dependency to its dependent until the dependency has been constructed. But it may have its own dependencies. So now there is a topological sorting problem. Turns out computers are really good at topological sorting. So, someone made the computer do it, and we call that a dependency injection container. Tada.
- gonzo41 5y agoDI containers don't stop circular dependencies. It's not a massive advantage.
- mayank 5y agoYes, but modern DI (like Dagger for Java) can detect cyclic dependencies at compile time and break the build.
- gonzo41 5y agoI know, but just thinking though what you're doing and building a program for right now, rather than building for a future that may never happen is probably going to deliver more success. I like constructor based dependency injection personally. It's simple and I don't really see a stack of value obsessing about the possibility of code reuse.
- contravariant 5y agoI can detect a cyclic reference before I even finish writing the code, what's the point of letting the compiler figure it out?
- mayank 5y ago> I can detect a cyclic reference before I even finish writing the code, what's the point of letting the compiler figure it out? The compiler generally has better attention to detail and the ability to deal with larger object graphs than the typical human.
- contravariant 5y agoSure, but how are you going to write code that uses a class with a circular dependency?
- KronisLV 5y agoCuriously i've actually seen people use @Lazy in Spring projects to allow for circular dependencies, to deal with situations where the services or their dependencies aren't structured like a neat tree with leafs, but rather some interdependent graph with cycles. Honestly, in practice it worked and wasn't too bad to work with, which was interesting to behold - in practice everyone talks about how circular dependencies are bad for a variety or reasons (trying to print or process data and ending up with endless loops comes to mind), but then there just was that system that was chugging along without a care in the world.
- choeger 5y agoIf you have so many top-level components or change your dependency graph that often that sorting becomes an issue you want to automate, you might want to reconsider your architecture.
- anothernewdude 5y agoAll you are doing is pushing the problem to runtime.
- Sankozi 5y agoTo any single integration test to be precise. So your builds fails few seconds later if something is wrong.
- valenterry 5y agoOf course the order matters - but you know what? It's good that this is explicit in the sourcecode and even checked by the compiler. Since now, I can go to the project's root and read step by step how the application is architected and build up and which are core dependencies and which are not. With dependency injection containers, everything is shuffled around and I'm now at the mercy of a library to tell me what I want to know.
- rallyaround 5y ago>you can't pass a dependency to its dependent until the dependency has been constructed. > But it may have its own dependencies. So now there is a topological sorting problem. I guess I just don't see the problem. That solves itself naturally through normal programming. Some function takes A as argument? Well I obviously make that A first. A requires B to setup? Well obviously I make that first. I don't need to "sort" anything, it just follows naturally from the types and constraints of the API.
- yen223 5y agoBest part is you haven't introduced new constructs or DSLs or XML configuration. It's just plain old Python functions calling other functions.
- garethrowlands 5y agoYes you're right, it works really really well.
- paskozdilar 5y ago> Consistently applied, all construction gets pushed to the entry-point of the program. Congratulations, you've just discovered the so-called "composition root". I never knew this has an explicit name. > Turns out computers are really good at topological sorting. So, someone made the computer do it, and we call that a dependency injection container. Tada. What exactly is a "dependency injection container"? I've searched the internet for the definition, but I've only gotten more confused by all the PHP and C#. You mention topological sorting of dependencies - does that assume a hierarchical object model or can it be used with basically anything? EDIT: From what I've read, it appears that a Dependency Injection Container is just an object that: 1) for each dependency type, has a `GetDependency: Type -> Object` method 2) lazily creates dependencies as required by the dependents and the dependency dependencies. Is this accurate?
- tored 5y agoAnd usually configurable, e.g. for interface Reader use FileReader implementation.
- ivanche 5y agoBeautiful article by Mark Seemann on composition root [1]. Another one on when to use DI container [2]. [1] https://blog.ploeh.dk/2011/07/28/CompositionRoot/ https://blog.ploeh.dk/2011/07/28/CompositionRoot/ [2] https://blog.ploeh.dk/2012/11/06/WhentouseaDIContainer/ https://blog.ploeh.dk/2012/11/06/WhentouseaDIContainer/
- SideburnsOfDoom 5y agoit is at heart quite simple. But this characteristic doesn't always mean that it has few benefits. Off the top of my head, things that DI enables include: * swapping out a class for an equivalent becomes an configuration concern. * whether a type is a singleton or transient becomes a separate concern to that class's implementation, and it can be changed in the app startup config code. There can be drawbacks too, if done badly, but I don't think that the argument that "this doesn't do very much" is entirely relevant to the pros and cons.
- necovek 5y agoWhile some configurability is desireable, being explicit is even more so. I wonder why Zope died out? It has everything, it's a completely sound design, and makes all components interchangeable, and it's very pretty when done properly. It died out because it's verbose and gets hairy quickly in the real world where not everyone is on the same page, so it never ends up being used "properly". Go for simpler and you have a higher chance of everybody getting the intent on their first reading of code. It's going to be hard to get people to not do DI "properly" if you simply ask them to pass dependencies in (though I am sure they still could by eg. passing classes vs instances in).
- bagels 5y agoSeems like the most minimalist. Maybe too minimalist though?
- zby 5y agoI really like that. Not just for testing - it makes the libraries much more flexible. But to make it a bit less burdensome for the most common usage patterns I like to relax it a bit with the possibility of building default object params in __init__.
- BerislavLopac 5y agoI usually provide two to three ways to inject dependencies, especially if an object needs multiple dependencies of the same type: 1. An add_foo instance method. 2. Arguments to the `__init__` method, ideally as *args, using add_foo internally. 3. Optionally, an @instance.foo decorator which inserts the dependency as it's being defined (again, using add_foo internally).
- iovrthoughtthis 5y agothis is the best way. everything else is incredibly over complicated and magic
- PartiallyTyped 5y agoWhat about builder patterns?
- necovek 5y agoI was going to propose the same thing with code: class MyDep: def do_foo(self): print("fooo!") class Fooser: def __init__(self, foo_dooer: MyDep): self.foo_dooer = foo_dooer def act(self): self.foo_dooer.do_foo() class FakeDep(MyDep): def do_foo(self): print("booo!") # With the grand unveiling: fooser = Fooser(MyDep()) Seems a lot simpler than 100 lines they brag about, yet this includes the business logic implementation for the code as well (dependency injection takes a couple of lines total; if you add interface handling or defaults, you'll add a few lines).
- nhumrich 5y ago> Oddly enough, dependency injection isn't that popular in Python Because you don't need dependency injection in python. You can mock out any object anywhere during your tests.
- jseban 5y ago> Because you don't need dependency injection in python. You can mock out any object anywhere during your tests. Exactly! I recommend anyone who comes in contact with dependency injection to really think about this.
- necovek 5y agoWhen discussing these in Python, you need to be careful to differentiate between "mocking" and "monkey patching", especially so because both of those are provided by Python's standard library "mock" module. Mocking, and specifically mocking with Python mock module, is a quick approach to construct faked objects that simply do not fail and which log any method execution on them. Dependency injection is usually done with more complete fakes, ideally with tested fakes (fakes which pass the same sets of tests as the original, "production" implementation does). Monkey-patching is what allows Python developers to avoid dependency injection altogether. It has its own set of risks (basically, it might affect things you are unaware of), though you can reduce those by patching exactly at the import place of your dependency (eg. patch("mymodule.utc_now") vs patch("datetime.datetime.utc_now") — if you are not doing that today, start now! :). Still, I consider readability the main concern when programming, with testability being second (no surprise they are closely related, as tests will document behaviour precisely). With Python's monkey-patching and mocking, you are at a large risk of producing illegible gibberish, both in tests and code. If you can at all avoid it, you should! Simple dependency injection by passing dependencies directly in (functions or classes) makes everything explicit and trivial to comprehend (and test). Unfortunately, the biggest problem is that the world of Python libraries is completely filled with magic side effects, so you sometimes can't effectively avoid doing either of those (it would be too costly).
- ptx 5y agoWhat about when there's some environment-specific (or otherwise problematic) dependency that you need to mock because it can't be imported where you want to run the unit tests? Patching it afterwards doesn't work when the import of the default implementation fails. If you patch it ahead of time by inserting a dummy module, that dummy module might be used to initialize other modules that you weren't targeting, so undoing the patch after the test might be very tricky. You can sort of make it work, but it seems really hacky, so for that reason I'm currently refactoring towards DI (without frameworks). But I'm open to better ideas.
- jmercouris 5y agoStop trying to make Python Java.
- NewMountain 5y ago100% this. Having worked on projects using and not using DI in several languages, I'd prefer a world in which we just say no to DI. What it offers, relative to the indirection and cognitive overhead has almost always been net negative.
- tored 5y agoHow is DI a cognitive overhead?
- mkl95 5y agoOP mentioned Python, and I agree that it's a cognitive overhead in that particular case, in the sense that it's an abstraction that can be avoided. If you are using something like Java or C++ then it's easy to see the benefits (some C++ developers would disagree though, read https://accu.org/journals/overload/25/140/pamudurthy_2403/ https://accu.org/journals/overload/25/140/pamudurthy_2403/).
- pydry 5y agoThe extra levels of indirection mean more code. Python programs are about 30-40% shorter than Java equivalents. More code = more cognitive overhead
- mkl95 5y agoYet another bloated pattern implementation that can be avoided by leveraging the standard library
- shaftoe444 5y agoEvergreen comment
- cabirum 5y agoDependency Injection does not require a DI Container. You know, like have an app_init.py and instantiate all services/dependencies manually; providing them as constructor args to other instances as needed. Use other implementations/mocks in tests setup or dev environments.
- kissgyorgy 5y agoIt bothers me very much that people who don't even understand the basics of Python want to teach complex concepts. This is why there are lots of misconceptions and terrible code in Python. __init__ is NOT the constructor in Python, the object is already created at the time when __init__ is started. Maybe this detail doesn't matter in this article, but still teaching the wrong thing.
- ptx 5y agoAt the risk of showing my ignorance... what makes Python's __init__ fundamentally different from Java's constructors? The latter also has access to the "this" reference with some of its state initialized and some not.
- teh_klev 5y agoI am not a Python expert, but I think strictly speaking __init__ is the object instance initialiser. The "constructor" is the __new__ method (which is a static method) which will create an instance of the class. Afterwards a call is made to the __init__ method on the new instance, passing any parameters specified in the __new__ method and the newly created instance (as self). As kissgyorgy rightly points out, it's the interpreter that performs these calls. Python is a bit more revealing about how the sausages are made. https://docs.python.org/3.10/reference/datamodel.html#object.__new__ https://docs.python.org/3.10/reference/datamodel.html#object... https://docs.python.org/3.10/reference/datamodel.html#object.__init__ https://docs.python.org/3.10/reference/datamodel.html#object...
- ptx 5y agoHmm, perhaps there is no such thing as a constructor in Python then? As the documentation says, "__new__() and __init__() work together in constructing objects". In Java's constructors you already have access to the instance (and could leak it outside the object if you wanted to) whereas in __new__ you don't, so it doesn't seem like __new__ by itself could be called the constructor either.
- a3w 5y agoThis does not seem to implement dependency inversion (DIP). (IIRC, Robert Martin states that DIP is one of the principles[1] on which dependency injection (DI) can be built as a pattern.) Since python does not have interfaces (unless you write classes that to nothing, except perhaps raise errors if methods are not overwritten) this does not seem to actually do DI/IoC completely. [1]: Together with DIP as the second principle. <Edit>Flipped IoC (e.g. factories) and DIP (interfaces for abstraction) which I seem to had memorized with swapped meanings</edit>
- BerislavLopac 5y agoPython has protocols. But I'm not sure if they relate to this directly.
- a3w 5y ago"new" in 3.8, oh nice! For static type checking, which might help even the small to medium sized scripting projects which I usually do.
- BerislavLopac 5y ago> "new" in 3.8, oh nice You can have them in earlier versions with the typing-extensions library. [0] > For static type checking Protocols can be used for runtime checks as well. [1] [0] https://pypi.org/project/typing-extensions/ https://pypi.org/project/typing-extensions/ [1] https://docs.python.org/3.9/library/typing.html#typing.runtime_checkable https://docs.python.org/3.9/library/typing.html#typing.runti...
- kissgyorgy 5y agoI don't understand the point of this implementation. If you have to call di.get("A") in __init__, why is it better than just instantiating A() directly? It's not even a dependency injection I guess, as the framework should take care of the automatic instantiation of objects without the object having to know even that a Di is in place.
- jseban 5y agoYeah it doesn't make any sense, the whole point of DI is to work around Java's static type system to enable mocking for testing, which is a problem you don't have outside of static typing. But the "why" around these design patterns have been long lost, and people will argue to death that the patterns should always be used in every language/runtime and come up with ever more vague and circular reasons until you can't argue against it.