4 ms·

I have a theory that auto-wired Dependency Injection in a single DI container is partly to blame for monolith spaghetti. Once an app reaches a certain size and

by scanr 5y ago

I have a theory that auto-wired Dependency Injection in a single DI container is partly to blame for monolith spaghetti. Once an app reaches a certain size and anything can depend on anything else, reasoning about the whole can become difficult. I think there is value in wiring a monolith together in such a way that each course grained subcomponent exposes a constrained interface into the rest of the system (payments, orders, shipping, customers etc) before needing to break it into distributed micro-services. Note: I quite like Dependency Injection, I just think the 1 giant bag of dependencies can lead to complexity at scale.

lmilcin 5y ago
I wouldn't blame (badly implemented) DI for all the problems of monoliths. I think the real main issue is lack of discipline when dividing the application into modules. Spaghetti is basically defined as an application where real modularisation does not exist and everything talks to everything. It is much easier to work with an application when you can abstract parts of it when you are solving your problem. You effectively work on much smaller part of the application. Spaghetti == you effectively have to take into account possibility of the piece of code you look at interacting with any other piece of code in the application. Well modularised application == you only need to take into account the contents of your current module and the interface of the other modules you are using. One reason why microservices sort of work (when done well) is because they force people to think about APIs and how those services talk to each other. In most cases you could just put these microservices as modules in a monolithic application and expend the effort on ensuring APIs and application structure. I have successfully rolled back couple microservice initiatives by integrating services into monoliths. This usually results in the team getting back a lot of their time because their environment suddenly became much simpler. Less applications to manage, less network communication, less possible ways for things to break, less frameworks, less resources needed to run the application, less processes (like processes around deployment, release management, etc.), less boilerplate code, and so on. The list is very long. Of course, when you work on a large monolith vs a lot of small microservices, it is now important to be able to structure your applications. But there is also an opportunity for improvement.
- jstimpfle 5y ago
  Ravioli == you have so many small distinct things on that are hard to stick together, which makes it hard to build a larger structure out of them.
  - lmilcin 5y ago
    Then try to not make the modules too small? Sure, if you were ready to make an entire separate application for it it will not be worse when you turn it into one package of a monolith? Also you can build nested structures with modules (like a larger module consisting of smaller submodules). Divide and conquer. This is how abstraction works in a nutshell. You start with a large problem, divide it into smaller parts (modules), each part you treat as a separate problem dividing it into smaller modules, and so on. There exists no need to have a flat application consisting of hundreds of sibling modules. An application might consist of larger modules, then some of those might have smaller submodules (packages), then those might have even smaller ones (classes), then you have methods, then you have statements, then you have function calls, etc. A large monolith does not have to succumb to spaghetti or ravioli-type structuring. A good developer should know how to structure an application of any size.
malort 5y ago
This is exactly what Shopify does with their monolithic Rails app [1]. I worked at a company with ~200 engineers that used the same general architecture and I really enjoyed it. We got a lot of the benefits (clear interfaces, teams able to work on their system without having to understand the whole platform, build optimization, etc) without any of the operational headaches that come with microservices. [1]: https://shopify.engineering/shopify-monolith https://shopify.engineering/shopify-monolith
mcv 5y ago
I notice there's a lot of comments here saying this exact same thing, and it's also what seems most sensible to me. Yet microservices are getting all the hype. Should be hype a thorough modular design more?
- heavyset_go 5y ago
  Yes, but microservices maximize cloud vendors' revenue compared to monoliths.
  - Nextgrid 5y ago
    They also maximize the number of engineers and managers needed to oversee the whole thing. People are unlikely to speak against something that guarantees their employment.
- truffdog 5y ago
  The nice thing about microservices is that you don't need iron discipline to maintain modular boundaries, instead its just how things work. I like to analogize it to assembly vs structured programming languages, or C vs Java- you can write the same programs in either, you can write bugs in either, but there are whole classes of pitfalls that get erased by moving from one language to another.
  - yakshaving_jgt 5y ago
    That’s not true. People in industry commonly refer to the consequence of their misguided decision to embark on a microservices adventure as a “distributed monolith”.
loevborg 5y ago
Check out Polylith for a description of this great pattern https://polylith.gitbook.io/polylith/ https://polylith.gitbook.io/polylith/
bob1029 5y ago
> Once an app reaches a certain size and anything can depend on anything else, reasoning about the whole can become difficult. I have experienced this pain so many times and in so many different varieties. Often, you cannot meaningfully subdivide the problem space without ruining the logical semantics per the business (i.e. bowl of spaghetti). An alternative is to embrace the reality that circular dependencies are actually inevitable and appropriate ways to model many things. The example scenario I like to use is that of a typical banking customer. One customer likely has a checking and savings account. For each of those accounts, there are potentially multiple customers (joint ownership). Neither of these logical business types will ever "win" in the real world DI graph. Certainly you can start to invent bullshit like AccountCustomer and CustomerAccount, but that only gets you 1 layer deeper into an infinitely-recursive rabbit hole of pain and suffering. There also exists the relational path, but I have heavily advocated for that elsewhere and it is not always applicable when talking about code-time BL method implementation. Being able to model things just as they are in the real world is a big key to success in the more complicated problem domains. Instead of trying to control what depends on what, I shifted my thinking to: > What needs to start up and in what order? Turns out, most things don't really care in practice. The only thing I have to explicitly initialize before everything else in my current code base is my domain model working set (recovery from snapshot/event logs) and settings. I decided to not use DI for any business services. Instead, all services become a static class with static members that can be invoked from anywhere. This also includes the domain model instance which is used as the in-memory working set. This type just contains an array of every subordinate domain type (Customers, Accounts, etc.). By having the working set available as a public static type, every service can directly access it without requiring method injection. If I was working with a different problem domain (or certain bounded context within this one), I might prefer method-level injection. Yes - according to every book on programming style you ever read, this is an abominable practice. Unit testing this would be difficult/impossible. But you know what? It works. It's simple. I can teach a novice how to add a new service in an hour. A project manager stumbling into AccountService might actually walk away enlightened. You can circularly-reference things at runtime if you need to. I've got some call graphs that bounce back and forth between customer & account services 5+ times. And it totally makes sense to model it that way too as far as the business is concerned. Everyone is happy.
4rb1t 5y ago
In theory it all sounds great and makes total sense but when the company grows to a billion dollar firm and hires engineers to work on the said monolith thats when things breakdown. The founding team was a closet knit team and ensure there is no spaghetti code but the moment you move on from that closely knit group its hard to enforce constraints. Have worked at multiple companies that started out as a monolith and are still running the monolith in some form or the other while breaking it down into micro services.
- deleted 5y ago
  [deleted]
throwaway894345 5y ago
I’ve never been part of a monolith that used a DI framework, but I’ve seen quite a few monoliths fail (as in “the project becomes too convoluted and iteration slows to a crawl until the project is canceled or effectively rewritten”) and I certainly believe that one important reason microservices do well is that they enforce the modularity that you describe. That said, a lot of critics of microservices describe similar issues of indiscernible chaos, so either I’ve been very fortunate or microservice critics are gaslighting us. :)
deleted 5y ago
[deleted]
rzzzt 5y ago
There was another article about building this structure into a single, self-contained executable and decide via command line arguments which piece(s) to use at startup. For some reason I remember "microlith", but it must be another clever word combination because I can not find any relevant HN posts, just one about archaeology... You can run a single copy of the resulting binary (eg. for testing) that spins up all sub-components, or copy it to multiple machines and start individual parts. The ones which happen to run together can use in-process communication as well, others will have to dial in via remoting.
- jdlshore 5y ago
  I’ve coined the term “microlith,” but I’m probably not the only one and it may not be what you’re thinking of. I wrote about it in the new edition of my book, and also discussed it here: https://www.jamesshore.com/v2/projects/lunch-and-learn/microservice-architecture-without-overhead https://www.jamesshore.com/v2/projects/lunch-and-learn/micro...
BatteryMountain 5y ago
At that point it becomes useful to look at some indirection/decoupling patterns like the Mediator pattern. Then instead of having a IService with 15 methods with a construction with 20 dependencies listed (multiply it with say 50 IServices in your project - mmmm spagheti project), you end up with a IHandler with a single method and a constructor that only has say 3 injected dependencies, only what it needs. The trade off is, you now have 100's of small Handler files, some people don't like that BUT you now have a pleasant git commit history per file and most of the code of a Handler file fits on one screen and it is easy to digest. You can also let a handler trigger other handlers if it needs to (via the same mediator). It also fits with SRP. Auto-wiring 95% of your dependencies still stays intact as the above plumbing will need it to work.
- ceesharp 5y ago
  If your IService has 15 methods and your ctor has 20 dependencies injected, then you have other issues. :)

by scanr 5y ago

lmilcin 5y ago

I wouldn't blame (badly implemented) DI for all the problems of monoliths. I think the real main issue is lack of discipline when dividing the application into modules. Spaghetti is basically defined as an application where real modularisation does not exist and everything talks to everything. It is much easier to work with an application when you can abstract parts of it when you are solving your problem. You effectively work on much smaller part of the application. Spaghetti == you effectively have to take into account possibility of the piece of code you look at interacting with any other piece of code in the application. Well modularised application == you only need to take into account the contents of your current module and the interface of the other modules you are using. One reason why microservices sort of work (when done well) is because they force people to think about APIs and how those services talk to each other. In most cases you could just put these microservices as modules in a monolithic application and expend the effort on ensuring APIs and application structure. I have successfully rolled back couple microservice initiatives by integrating services into monoliths. This usually results in the team getting back a lot of their time because their environment suddenly became much simpler. Less applications to manage, less network communication, less possible ways for things to break, less frameworks, less resources needed to run the application, less processes (like processes around deployment, release management, etc.), less boilerplate code, and so on. The list is very long. Of course, when you work on a large monolith vs a lot of small microservices, it is now important to be able to structure your applications. But there is also an opportunity for improvement.

jstimpfle 5y ago
Ravioli == you have so many small distinct things on that are hard to stick together, which makes it hard to build a larger structure out of them.
- lmilcin 5y ago
  Then try to not make the modules too small? Sure, if you were ready to make an entire separate application for it it will not be worse when you turn it into one package of a monolith? Also you can build nested structures with modules (like a larger module consisting of smaller submodules). Divide and conquer. This is how abstraction works in a nutshell. You start with a large problem, divide it into smaller parts (modules), each part you treat as a separate problem dividing it into smaller modules, and so on. There exists no need to have a flat application consisting of hundreds of sibling modules. An application might consist of larger modules, then some of those might have smaller submodules (packages), then those might have even smaller ones (classes), then you have methods, then you have statements, then you have function calls, etc. A large monolith does not have to succumb to spaghetti or ravioli-type structuring. A good developer should know how to structure an application of any size.

malort 5y ago

This is exactly what Shopify does with their monolithic Rails app [1]. I worked at a company with ~200 engineers that used the same general architecture and I really enjoyed it. We got a lot of the benefits (clear interfaces, teams able to work on their system without having to understand the whole platform, build optimization, etc) without any of the operational headaches that come with microservices. [1]: https://shopify.engineering/shopify-monolith https://shopify.engineering/shopify-monolith

mcv 5y ago

I notice there's a lot of comments here saying this exact same thing, and it's also what seems most sensible to me. Yet microservices are getting all the hype. Should be hype a thorough modular design more?

heavyset_go 5y ago
Yes, but microservices maximize cloud vendors' revenue compared to monoliths.
- Nextgrid 5y ago
  They also maximize the number of engineers and managers needed to oversee the whole thing. People are unlikely to speak against something that guarantees their employment.
truffdog 5y ago
The nice thing about microservices is that you don't need iron discipline to maintain modular boundaries, instead its just how things work. I like to analogize it to assembly vs structured programming languages, or C vs Java- you can write the same programs in either, you can write bugs in either, but there are whole classes of pitfalls that get erased by moving from one language to another.
- yakshaving_jgt 5y ago
  That’s not true. People in industry commonly refer to the consequence of their misguided decision to embark on a microservices adventure as a “distributed monolith”.

loevborg 5y ago

Check out Polylith for a description of this great pattern https://polylith.gitbook.io/polylith/ https://polylith.gitbook.io/polylith/

bob1029 5y ago

> Once an app reaches a certain size and anything can depend on anything else, reasoning about the whole can become difficult. I have experienced this pain so many times and in so many different varieties. Often, you cannot meaningfully subdivide the problem space without ruining the logical semantics per the business (i.e. bowl of spaghetti). An alternative is to embrace the reality that circular dependencies are actually inevitable and appropriate ways to model many things. The example scenario I like to use is that of a typical banking customer. One customer likely has a checking and savings account. For each of those accounts, there are potentially multiple customers (joint ownership). Neither of these logical business types will ever "win" in the real world DI graph. Certainly you can start to invent bullshit like AccountCustomer and CustomerAccount, but that only gets you 1 layer deeper into an infinitely-recursive rabbit hole of pain and suffering. There also exists the relational path, but I have heavily advocated for that elsewhere and it is not always applicable when talking about code-time BL method implementation. Being able to model things just as they are in the real world is a big key to success in the more complicated problem domains. Instead of trying to control what depends on what, I shifted my thinking to: > What needs to start up and in what order? Turns out, most things don't really care in practice. The only thing I have to explicitly initialize before everything else in my current code base is my domain model working set (recovery from snapshot/event logs) and settings. I decided to not use DI for any business services. Instead, all services become a static class with static members that can be invoked from anywhere. This also includes the domain model instance which is used as the in-memory working set. This type just contains an array of every subordinate domain type (Customers, Accounts, etc.). By having the working set available as a public static type, every service can directly access it without requiring method injection. If I was working with a different problem domain (or certain bounded context within this one), I might prefer method-level injection. Yes - according to every book on programming style you ever read, this is an abominable practice. Unit testing this would be difficult/impossible. But you know what? It works. It's simple. I can teach a novice how to add a new service in an hour. A project manager stumbling into AccountService might actually walk away enlightened. You can circularly-reference things at runtime if you need to. I've got some call graphs that bounce back and forth between customer & account services 5+ times. And it totally makes sense to model it that way too as far as the business is concerned. Everyone is happy.

4rb1t 5y ago

In theory it all sounds great and makes total sense but when the company grows to a billion dollar firm and hires engineers to work on the said monolith thats when things breakdown. The founding team was a closet knit team and ensure there is no spaghetti code but the moment you move on from that closely knit group its hard to enforce constraints. Have worked at multiple companies that started out as a monolith and are still running the monolith in some form or the other while breaking it down into micro services.

deleted 5y ago
[deleted]

throwaway894345 5y ago

I’ve never been part of a monolith that used a DI framework, but I’ve seen quite a few monoliths fail (as in “the project becomes too convoluted and iteration slows to a crawl until the project is canceled or effectively rewritten”) and I certainly believe that one important reason microservices do well is that they enforce the modularity that you describe. That said, a lot of critics of microservices describe similar issues of indiscernible chaos, so either I’ve been very fortunate or microservice critics are gaslighting us. :)

deleted 5y ago

[deleted]

rzzzt 5y ago

There was another article about building this structure into a single, self-contained executable and decide via command line arguments which piece(s) to use at startup. For some reason I remember "microlith", but it must be another clever word combination because I can not find any relevant HN posts, just one about archaeology... You can run a single copy of the resulting binary (eg. for testing) that spins up all sub-components, or copy it to multiple machines and start individual parts. The ones which happen to run together can use in-process communication as well, others will have to dial in via remoting.

jdlshore 5y ago
I’ve coined the term “microlith,” but I’m probably not the only one and it may not be what you’re thinking of. I wrote about it in the new edition of my book, and also discussed it here: https://www.jamesshore.com/v2/projects/lunch-and-learn/microservice-architecture-without-overhead https://www.jamesshore.com/v2/projects/lunch-and-learn/micro...

BatteryMountain 5y ago

At that point it becomes useful to look at some indirection/decoupling patterns like the Mediator pattern. Then instead of having a IService with 15 methods with a construction with 20 dependencies listed (multiply it with say 50 IServices in your project - mmmm spagheti project), you end up with a IHandler with a single method and a constructor that only has say 3 injected dependencies, only what it needs. The trade off is, you now have 100's of small Handler files, some people don't like that BUT you now have a pleasant git commit history per file and most of the code of a Handler file fits on one screen and it is easy to digest. You can also let a handler trigger other handlers if it needs to (via the same mediator). It also fits with SRP. Auto-wiring 95% of your dependencies still stays intact as the above plumbing will need it to work.

ceesharp 5y ago
If your IService has 15 methods and your ctor has 20 dependencies injected, then you have other issues. :)