25 ms·

Microservices was always a solution to the organisational problem of getting more developers working on a system at once. A lot of people working on one code ba

Microservices was always a solution to the organisational problem of getting more developers working on a system at once. A lot of people working on one code base over the top of each other often causes problems and issues with ownership. The solution was services independently developed with the trade off cost being increased complexity on a range of things. There is an essay in the mythical man month about Conways law and microservices are very much about a way to use Conways law to achieve scale of development. You likely don't need microservice until you hit a scale where the monolith is a real pain point. You can probably cope with a monolith especially with some reasonable design effort up to the 100 developers maybe more with a lot of separation but at some point in the >25 developers range it becomes cheaper and easier to switch to multiple deployed units of software and deal with the increased system complexity and interface designs and the inherent versioning issues it causes. It is easier to start with a monolith, find the right design and then split on the boundaries than it is to make the correct microservices to begin with.

gowld 4y ago
> Microservices was always a solution to the organisational problem of getting more developers working on a system at once. Hmm? You can do that simply with API boundaries inside a program. Linux kernel has huge development team that compiles a all binary.
- jayd16 4y ago
  You can use microservices to decouple things like deployment schedules, build dependencies, configurations, etc.
  - breakalot 4y ago
    You mean "duplicate"?
- CogitoCogito 4y ago
  But that requires discipline.
  - peterangular 4y ago
    Discipline, leadership, organization, and, most of all: coherency between developers. Most orgs do not have these things - especially not at the level of the Linux team. Also the motives of OSS are completely different than that of your typical business - I personally think this keeps OSS team coherent/productive for far longer than your typical corp can.
- gjulianm 4y ago
  I think the Linux kernel is a special case. For starters, due to the nature of the kernel itself, a lot of components are very intertwined. You'll be hard pressed to find one area of the kernel core in which you can work without at least having an idea of how other parts work. But for other components, such as drivers, you'll find they're far more independent and different teams do work on them, and update them separately even in out-of-tree module builds, mimicking a bit how you'd do microservices in that limited setting.
  - vlunkr 4y ago
    So is every piece of large software a 'special case'? Besides the Linux Kernel, there's every other OS ever written, office tools, browsers, video games, all sorts of SaaS apps. The idea that microservices solve some problem in that space is bogus.
    - gjulianm 4y ago
      It is a special case because there's nothing underneath the kernel. Different pieces of the kernel usually cannot interact through anything that isn't the single binary, there aren't more levels of abstraction. But office tools, browser, video games, SaaS apps can benefit from the architecture. Microservices, at least for me, doesn't mean "different processes running in containers on AWS", but separate "services" that can be run and deployed independently. One easy example is login components, say any of those tools offers the possibility to the user to login with one or more remote services. Instead of putting that code in the same monolith, you could have a separate binary that gets called by the parent and spits out an authorization token. The separate binary can be tested, developed and updated separately. Of course, you can't apply microservices to everything and have it make sense, but the ideas of separate tools deployed and updated independently are not bogus. I mean, the Unix philosophy could be understood as a kind of microservices architecture, with different independent tools managed by separate teams.
      - vlunkr 4y ago
        > the ideas of separate tools deployed and updated independently are not bogus No, but the idea that you should do that because you have a big team is bogus. There are good reasons to break some applications up into services, and this is the least compelling one.
- jimbokun 4y ago
  Mainly because for a very long time, Linus reviewed every pull request.
- theamk 4y ago
  Sure you can, and then you have a team that introduced a bug in common library that caused 100% CPU consumption on all cores, and tests did not caught it so somehow got into production. In the monorepo world, your whole system is broken, it is all hands on board to try to fix this, everyone is stressed out. In the microservice world, you have only one microservice which went down, so most teams don't have to worry.. in the worst case, they'll say: "Sorry, but we depend on service X and they are down.. blame them, nothing we can do". Sure, that team which introduced the bug is stressed, but the average company stress is much lower. Having a successful monorepo requires organized, cohesive team with good communications -- or at least a team with a highly experienced people with veto power (this is the Linux kernel model). Unfortunately, a lot of real-life businesses do not have it.
  - hakre 4y ago
    > In the monorepo world, your whole system is broken, it is all hands on board to try to fix this, everyone is stressed out. No, not at all. And the kernel is a single repo even. > In the microservice world, you have only one microservice which went down, so most teams don't have to worry.... blame them, nothing we can do Well how is that better than just switching the dependency back to last known version and ship instead being dependent on a whole different team just to get dependencies fixed _and_ running. > Unfortunately, a lot of real-life businesses do not have it. That may be true, however benefit of those properties are in different kind of development projects, and this is not the question of whether monolith or microservices IMHO. Also not mono repo or non-source binary distribution.
  - seti0Cha 4y ago
    > In the microservice world, you have only one microservice which went down, so most teams don't have to worry.. My experience has been that one service going down (or even running slowly) can lead to cascading failures where identifying the root cause is a slow painful process. I know that in a well designed system that doesn't happen, but that's the nature of all bugs, isn't it?
    - theamk 4y ago
breakalot 4y ago
> Microservices was always a solution to the organisational problem of getting more developers working on a system at once. Wasn't Continuous Integration (CI servers) created long ago to solve that? Where developers can test and release their own changes, incrementally, without stepping on each other toes? What is next?
- Espressosaurus 4y ago
  CI helps releasing fast, it doesn't do anything about not stepping on each other's toes. You can imagine a case where two people are working on the same part of the codebase and having merge conflicts everywhere. CI doesn't help with that, though it does make it a little bit less stressful to resolve those conflicts. Microservices forces boundaries, which in turn allows you to scale the number of teams to the number of service boundaries and now you're no longer stepping on each other. With the disclaimer that I'm an embedded programmer so I don't have a dog in this fight, my reading of the literature suggests it's best seen as an organizational tool rather than a performance tool.
  - TheCoelacanth 4y ago
    > In software engineering, continuous integration (CI) is the practice of merging all developers' working copies to a shared mainline several times a day. [1] That certainly does help with avoiding conflicts. There are only so many conflicts that you can pick up in a few hours of work. [1] https://en.wikipedia.org/wiki/Continuous_integration https://en.wikipedia.org/wiki/Continuous_integration
anon23anon 4y ago
Feel like the conversation went something like this - - We need this thing done by <insert date> -- That's impossible - What if we hire more ppl? -- They'd just step over each other - Let's divide things up then -- ok * six months pass by * - why isn't it done yet? -- we had to onboard and train all those new hires which sucked up a lot of dev time plus the new architecture added complexity and increased communication costs - How can we make our onboarding process faster? * sigh and the cycles repeats it self
- aloisdg 4y ago
  "What one programmer can do in one month, two programmers can do in two months." Fred Brooks
chronofar 4y ago
> Microservices was always a solution to the organisational problem of getting more developers working on a system at once. I’m not sure that that’s entirely true. To be sure that’s an aspect, but it’s also conceptually quite reasonable to desire the best tool for a particular job rather than employ a kitchen sink for everything. Separation of concerns is a commonly admired quality in code, and it makes sense sometimes to have this separation be at the repo level. I don’t jive with the “only organizational not technical” characterizations this thread is rife with. It’s now hipster to say the cool shiny thing is lame, and it blinds some utility. > It is easier to start with a monolith, find the right design and then split on the boundaries than it is to make the correct microservices to begin with. This I definitely agree with in many, perhaps most, cases. But of course inertia, especially in organizations, could have future you lamenting such a decision you now have no easy way out of.
hintymad 4y ago
Not sure if this story is true: Okta used to have a monolith architecture on AWS. All their 300 engineers used to work on the same service, so they had to run all their tests for every commit. The cost? $60 per commit.
- cellis 4y ago
  Sounds pretty cheap, relative to 300 engineers.
  - samwillis 4y ago
    Assuming a dev is committing (ball park) 4 times per day, that's in the region of $55k/year per developer, or $16.5M/year all in. Thats not cheap...
    - djur 4y ago
      I would not assume that 300 developers are all committing 4 times a day. That seems very high.
      - jimbokun 4y ago
        And there you have it...$60 per commit means slowing down your commit velocity! Which means its longer before changes get tested or reviewed or deployed. The cost (money and more importantly time) of running tests fundamentally changes the development process.
        xorcist 4y ago
        More commits are not better.
        JMS2021 4y ago
        Unless you are a PHP developer
    - 4y ago
staticassertion 4y ago
> Microservices was always a solution to the organisational problem of getting more developers working on a system at once. I don't really agree. It's probably the most agreed upon benefit, but there are others. Fundamentally, microservices allow you to isolate state. I can take two services and split them up - and now I've split their state spaces. I can put a queue service between them and now I've sliced out the state of their communication. This slicing up of state has a ton of benefits if done right. 1. Isolation of state is basically the key to having scalable concurrency. Watch the Whatsapp talk on Erlang and they say "Isolation, Isolation, Isolation". 2. Isolation of services is great for security. You can split up permissions across your services, limit their access in a more granular way, etc. Those are pretty nice wins. They're achievable through discipline in a monolith (heavy use of module boundaries, heavy use of immutability - something most mainstream languages don't encourage) but a network boundary really forces these things and makes unintentional stateful coupling a lot more painful. > It is easier to start with a monolith, find the right design and then split on the boundaries than it is to make the correct microservices to begin with. I disagree. Starting with a bad set of microservices can be fixed by merging. Merging two codebases is trivial compared to splitting. Again, if I have isolation between the services, even if the slicing up was done badly, even if they're coupled, I can just remove the layer between them. Splitting has to start from a place of coupling and then try to decouple - this is especially hard with languages that encourage encapsulated mutable state (most of them).
- manigandham 4y ago
  Network boundaries cause far more problems than they solve, and you've just shifted the complexity to now securing the network, usually with even more additional services, proxies, services meshes, firewalls, etc.
  - staticassertion 4y ago
    > Network boundaries cause far more problems than they solve, They cause exactly 0 extra problems. A call from function A to function B can fail due to B having a bug. A call from service A to service B can fail due to B having a bug or a network failure. Either way, failure is possible and has to be handled - the network only makes that more obvious. Further, a call between functions can cause mutated shared state - not the case across a boundary, they physically do not share mutable state. > and you've just shifted the complexity to now securing the network Not really. Fundamentally you have split your service capabilites up - now you can apply least privilege as you desire.
dec0dedab0de 4y ago
Microservices was always a solution to the organisational problem of getting more developers working on a system at once. It may be a solution, but there were already solutions for that. As far as I remember, the orignal argument of microservices was to allow different teams to use whatever languages/tools they choose, as long as they provided an API. As opposed to having IT, payroll, and sales all having to wait for one development team to make everything. Then with docker and containers it became that you could isolate deployments, and scale out just the parts that need it. Which is great if you need to minimize outages, and have some parts of your system that get major spikes in traffic, and you decide it's worth the extra complexity.
- blantonl 4y ago
  This is where micro-services come in for me. Except, for all intents and purposes I'm a one man development shop. APIs. I've deployed about 10 or micro-services for use cases where a simple API is required for my applications. Maybe it's literally querying a row in a table and returning the result in JSON format. Or maybe it's to generate a JWT to access Apple Maps. Just craft it up in NodeJS, deploy to Lambda, and those little services haven't changed in years but they just keep cranking away. We can have both monoliths and micro-services where appropriate.
  - deleted 4y ago
    [deleted]
  - mason55 4y ago
    At the risk of committing a "No True Scotsman" fallacy, what you're talking about is not really what people are talking about when they are discussing the tradeoffs of a microservices architecture. While "microservice" obviously literally means a small service, when people talk about microservice architecture they're usually talking about building out applications that look like a single service to the end-user but in reality are a whole bunch of small services that talk to each other. Just creating a small service that exposes a small piece of functionality is clearly the right thing to do in many cases, but that's not what the discussion about microservices architectures is really about.
matchagaucho 4y ago
But also, according to Conway's Law, you end up with a product with leaky abstractions that reveal the underlying micro-services and org structure.
jbluepolarbear 4y ago
A better solution would be to build modules independently and have a single service use each module. Then things are unified, each team has a client it must deliver to, and issues must be fixed in order for them to be released in the final product. How are microservices better? Often each service is out of sync with each other, one service will upgrade and break another, that’s not possible with a single source and multiple modules.
- pixl97 4y ago
  Do people not version their API requests?
  - jbluepolarbear 4y ago
    Yep and it gets worse, you’ll have teams add new breaking features or deprecate features without letting the other team know. Hey their tests pass so what do they care. I’ve worked with both strategies and modules over microservices are easier to integrate, ensure integrity, and reduces the probability that stuff breaks in production. Often microservices develop to a set of requirements and never consider the client that will be using that service. With modules that can’t happen.
strictfp 4y ago
Organizing around software components is such an idiotic idea. Good luck changing your design or making end-to-end features when you have teams protecting "their" part of the software. You've just made sure that any significant change to the system requires coordinating many different teams, making it at least one order more difficult than before, and a lot more bureaucratic.
- peterangular 4y ago
  For my anecdotal experience, microservices is the introduction of fiefdoms in development. And, I get it. Selfishly I would love to come onto a project and only have to worry about net-new development. It is hard to learn, and then positively contribute within another engineered system... of any complexity. When it's "my code" I tend to be much more productive, and it's easier to find my way through the forest. Anymore, everyone is on a power-grab for "my tools" and "my way" in regards to everything they're working on with near-zero attempt to understand the base needs of the business/feature. Microservices fill this need well as devs can work with their preferred toolset and just setup I/O boundaries for everyone else to use their service. Also, this is good for resume-driven development as you can often claim creation/ownership over some "piece" which looks way better on paper vs. "contributed to a big 'ol project." I'm not against microservices, I'm just being honest with how I've seen them socially play out - all of this is anecdotal to my experiences.
  - dragonwriter 4y ago
    > For my anecdotal experience, microservices is the introduction of fiefdoms in development. Fiefdoms, whether implicit or explicit, are pervasive in organizations beyond trivial size; IME, software architecture choices may impact the shape of the fiefdoms, but not their existence.
- deleted 4y ago
  [deleted]
billiam 4y ago
>It is easier to start with a monolith, find the right design and then split on the boundaries than it is to make the correct microservices to begin with. Not pleasant, but true.
papito 4y ago
Dropbox - monolith Instagram - monolith StackOverflow - monolith (at least until SO Enterprise) There are many others. There is a reason why we avoided "distributed systems" in the past. It's insanely hard. Microsevices solve some problems in theory but create actual problems in practice. For most companies out there, distributed systems solve problems the companies don't even have. But, boy, do they need 10x more engineers to maintain this all.
- hakre 4y ago
  An they pose(d) some attraction. A boss after getting tired developing the baby that turned into a monolith and brought enough wealth to the table started to outline a new system with 12 or so microservices. Each one could be designed and planned and all written out in nice 2 to 6 page documents - one for each service. Now guess the size of the code wrangler team. And as this was so super-extravagant, guess the number of developers who were "promoted" to work on those micro-services. > But, boy, do they need 10x more engineers to maintain this all. Well stupidity saves you here. Ignore the one team per microservice rule and swap it with the "one-size-fits-it-all" developer rule: just promote one of the single team to be responsible for all micro-services. One lunatic that pushes code fast enough that management does not realize what is going on. Once everything is up and running, it just runs. The rest is downtime eventually, but you know what, its a software problem, what could be done? As long as everyone involved looks busy - and it's easy with plenty of micro-services to play whack-a-mole, bring one up again, kill two others, it is just a fail in the design as they should have been designed with failover but that was missed!
- bin_bash 4y ago
  Dropbox migrated to Go microservices
  - papito 4y ago
    Fair enough. At that scale it's different. My point is that many major companies were a productive team working with a monolith until it simply was not feasible.
ChrisCinelli 4y ago
Microservices are not to scale organization even if some organizations used them in this way. There is a big problem with this way of scaling. The sense ownership becomes the sense of defend your castle (team's microservice code) from invasion (external developer that are trying to add features or fix bugs). Microservices were also presented as a solution for encapsulation. But if you cannot write good encapsulated code in a monolith, what makes you think microservices are the answer? http://www.codingthearchitecture.com/2014/07/06/distributed_big_balls_of_mud.html http://www.codingthearchitecture.com/2014/07/06/distributed_... Microservices (or just services) are the solution of having more bandwidth for a set of features without having to more of everything else and preventing some part of the code to influence negatively some others.
WalterBright 4y ago
Analogy is in programming itself - divide up a program into modules that each have a very clearly defined interface. Then people can work in parallel on it. I know everybody agrees with this, and it is stupidly obvious. But time after time, I see source file after source file importing (or #include'ing) every other module in the project. A pull request to fix a problem affects 10 files instead of one. It seems to me that people are simply terrible at modularizing code. (Including me.) I've seen this problem far more often than not, in every organization I've worked with, for my entire career. Just the other day I worked on a function that did some Windows API calls for file I/O, allocated a bunch of memory, and processed the file contents. I split it up so the code that needed the Windows API did just that - it didn't allocate any memory, and the processing was done with a callback. That meant the Windows API accessing code didn't need to import anything from the rest of the program, and the rest didn't need to import the Windows API. (There are some success stories. The popularization of generic algorithms and ranges (or iterators) has been a big improvement. And the use of global variables has become frowned upon.)
- h2odragon 4y ago
  > people are simply terrible at modularizing code. Once you realize you needed it, its too late. Then once you've been burnt on that fire for a bit, you go all out in designing the next few projects with framework and forethought and wind up with 100 lines of that for a 10 line script. the moral is "life's a bitch"
  - 1penny42cents 4y ago
    That oscillation is necessary to be able to converge on the optimal medium for any given problem. Otherwise we get stuck in "modularization bad" or "modularization good" mindsets, where both are wrong.
- dkarl 4y ago
  I agree; people are much more careful about splitting responsibilities and designing APIs between processes (REST, GraphQL, XML-RPC) than between code modules. Simply switching from a monolith to microservices makes them more thoughtful about the consequences of design.
dragonwriter 4y ago
> Microservices was always a solution to the organisational problem of getting more developers working on a system at once. Microservices was always a solution to the problem that SOA got swallowed by “implementing XML heavy standards” and lost attention to it's architectural principles. (SOA was originally, and microservices a return to, a solution to reusing components and isolating faults so you need fewer, not more, developers and support personnel dedicated to any given system or collection of systems.)
outworlder 4y ago
> Microservices was always a solution to the organisational problem of getting more developers working on a system at once In other words, it's a great way to ship your org chart. I've only ever seen a handful of examples where the microservice boundaries were set regardless on how the actual teams are organized in the reporting structure.
- icambron 4y ago
  You'll ship your org chart whether you try to or not. There are two questions: a) will you architect your software to make this process (relatively) efficient? and b) to what degree are you willing to change your org chat to improve your architectural boundaries?
  - mwcampbell 4y ago
    I think one can control the degree to which software architecture reflects the org chart. To take an extreme example, Chrome is primarily a single, giant (~200 MB) binary. I'm sure the layout of the source repositories reflects the Chrome org chart. But thankfully, they always had the engineering discipline to produce the architecture that was best for performance (a monolith, albeit a multi-process one) without letting it get out of control.
logifail 4y ago
> It is easier to start with a monolith, find the right design and then split on the boundaries [..] This reminds me of the discussion on let pedestrians define the walkways[0] [1] "As time goes on, we get smarter. We learn more about ourselves or our customers — what we or they really want. Therefore, we’re at our dumbest at the beginning, and at our smartest at the end. So when should you make decisions? When you have the most information, when you’re at your smartest: as late as possible." [0] https://sive.rs/walkways https://sive.rs/walkways [1] https://www.theguardian.com/cities/2018/oct/05/desire-paths-the-illicit-trails-that-defy-the-urban-planners https://www.theguardian.com/cities/2018/oct/05/desire-paths-...
pbreit 4y ago
Where I've found "microservices" to be sensible is where you have multiple different clients to power (ie, website, iOS & Android). They all consumer the same JSON and render natively. Otherwise, microservices just turn what was once easy into very complicated.
lamontcg 4y ago
A technology reason to have services is that you can have large RAM/SSD caches on services and move that data off of your front end monoliths, shrinking the footprint of the monolith and keeping more data accessible quickly (although small caches for the very hottest data might be good to retain on the first tier). That data probably shouldn't be peeled off though until there's a software team there to maintain the new service, and it should really be living in its own data pool on the backend (you shouldn't be writing a service to front a single database table, you should be writing a service to front an entire database). If the data is large enough and important enough to be moved to a service, and there's performance wins by doing so, and if there's a team that can manage that service, then it probably is a good idea. A single team that is managing a monolith busting it up into a dozen services that they still manage, and all the data comes out of a single database server is probably the wrong way to do it. There might be some other cases where you have hard business requirements and such for some software to be able to ship without other software and need to build clean lines and services so features can be extracted and probably some other edge cases, but the larger point should be that extracting the service should almost be demanded, both technologically and organizationally. If you have a choice and it isn't clear, then a service is probably the wrong thing. Services also really shouldn't be "micro" until you're an absolutely huge company and you can burn entire teams on small edge cases.
jacobsenscott 4y ago
> A lot of people working on one code base over the top of each other often causes problems and issues with ownership. Microservices don't fix this. If I'm working on a feature in a monolith and I need to modify code someone else is also working on for some other feature that same situation would occur in a microservices based app. Just because the code is in a different git repo, and called with via an RPC doesn't change the fact that you need to change it. Most software is monolithic, and most critical software is millions lines of code that hundreds or thousands of people have worked on - all without using microservices.
deleted 4y ago
[deleted]
mbrodersen 4y ago
Linux is a monolith. With more than 25 developers I believe. Seems to be doing OK last time I looked.
mbrodersen 4y ago
There is this amazing technology called “libraries”, “modules”, and “frameworks” that allows thousands of developers to work on the same monolith. It has been around for quite a while. No micro-services required. There is even this amazing technology called “automatic testing” that can test each library/framework independently before delivering to clients. I happens to use both of those amazing technologies at work every day and it works really well. I also happen to be sharing an office with a college who is fighting a 20 years old micro-services application. The worst piece of garbage I have ever seen in my 25+ years career. Every little functionality is running in its own process. There are hundreds of them. The complexity and difficulty of the system is truly mind blowing. If you think monoliths are hard, just wait until you split the same functionality into a distributed collection of micro-services. Welcome to the world of distributed systems. The additional failure scenarios just sky rocketed. Have a nice day!

gowld 4y ago

> Microservices was always a solution to the organisational problem of getting more developers working on a system at once. Hmm? You can do that simply with API boundaries inside a program. Linux kernel has huge development team that compiles a all binary.

jayd16 4y ago
You can use microservices to decouple things like deployment schedules, build dependencies, configurations, etc.
- breakalot 4y ago
  You mean "duplicate"?
CogitoCogito 4y ago
But that requires discipline.
- peterangular 4y ago
  Discipline, leadership, organization, and, most of all: coherency between developers. Most orgs do not have these things - especially not at the level of the Linux team. Also the motives of OSS are completely different than that of your typical business - I personally think this keeps OSS team coherent/productive for far longer than your typical corp can.
gjulianm 4y ago
I think the Linux kernel is a special case. For starters, due to the nature of the kernel itself, a lot of components are very intertwined. You'll be hard pressed to find one area of the kernel core in which you can work without at least having an idea of how other parts work. But for other components, such as drivers, you'll find they're far more independent and different teams do work on them, and update them separately even in out-of-tree module builds, mimicking a bit how you'd do microservices in that limited setting.
- vlunkr 4y ago
  So is every piece of large software a 'special case'? Besides the Linux Kernel, there's every other OS ever written, office tools, browsers, video games, all sorts of SaaS apps. The idea that microservices solve some problem in that space is bogus.
  - gjulianm 4y ago
    It is a special case because there's nothing underneath the kernel. Different pieces of the kernel usually cannot interact through anything that isn't the single binary, there aren't more levels of abstraction. But office tools, browser, video games, SaaS apps can benefit from the architecture. Microservices, at least for me, doesn't mean "different processes running in containers on AWS", but separate "services" that can be run and deployed independently. One easy example is login components, say any of those tools offers the possibility to the user to login with one or more remote services. Instead of putting that code in the same monolith, you could have a separate binary that gets called by the parent and spits out an authorization token. The separate binary can be tested, developed and updated separately. Of course, you can't apply microservices to everything and have it make sense, but the ideas of separate tools deployed and updated independently are not bogus. I mean, the Unix philosophy could be understood as a kind of microservices architecture, with different independent tools managed by separate teams.
    - vlunkr 4y ago
      > the ideas of separate tools deployed and updated independently are not bogus No, but the idea that you should do that because you have a big team is bogus. There are good reasons to break some applications up into services, and this is the least compelling one.
jimbokun 4y ago
Mainly because for a very long time, Linus reviewed every pull request.
theamk 4y ago
Sure you can, and then you have a team that introduced a bug in common library that caused 100% CPU consumption on all cores, and tests did not caught it so somehow got into production. In the monorepo world, your whole system is broken, it is all hands on board to try to fix this, everyone is stressed out. In the microservice world, you have only one microservice which went down, so most teams don't have to worry.. in the worst case, they'll say: "Sorry, but we depend on service X and they are down.. blame them, nothing we can do". Sure, that team which introduced the bug is stressed, but the average company stress is much lower. Having a successful monorepo requires organized, cohesive team with good communications -- or at least a team with a highly experienced people with veto power (this is the Linux kernel model). Unfortunately, a lot of real-life businesses do not have it.
- hakre 4y ago
  > In the monorepo world, your whole system is broken, it is all hands on board to try to fix this, everyone is stressed out. No, not at all. And the kernel is a single repo even. > In the microservice world, you have only one microservice which went down, so most teams don't have to worry.... blame them, nothing we can do Well how is that better than just switching the dependency back to last known version and ship instead being dependent on a whole different team just to get dependencies fixed _and_ running. > Unfortunately, a lot of real-life businesses do not have it. That may be true, however benefit of those properties are in different kind of development projects, and this is not the question of whether monolith or microservices IMHO. Also not mono repo or non-source binary distribution.
- seti0Cha 4y ago
  > In the microservice world, you have only one microservice which went down, so most teams don't have to worry.. My experience has been that one service going down (or even running slowly) can lead to cascading failures where identifying the root cause is a slow painful process. I know that in a well designed system that doesn't happen, but that's the nature of all bugs, isn't it?
  - theamk 4y ago

breakalot 4y ago

> Microservices was always a solution to the organisational problem of getting more developers working on a system at once. Wasn't Continuous Integration (CI servers) created long ago to solve that? Where developers can test and release their own changes, incrementally, without stepping on each other toes? What is next?

Espressosaurus 4y ago
CI helps releasing fast, it doesn't do anything about not stepping on each other's toes. You can imagine a case where two people are working on the same part of the codebase and having merge conflicts everywhere. CI doesn't help with that, though it does make it a little bit less stressful to resolve those conflicts. Microservices forces boundaries, which in turn allows you to scale the number of teams to the number of service boundaries and now you're no longer stepping on each other. With the disclaimer that I'm an embedded programmer so I don't have a dog in this fight, my reading of the literature suggests it's best seen as an organizational tool rather than a performance tool.
- TheCoelacanth 4y ago
  > In software engineering, continuous integration (CI) is the practice of merging all developers' working copies to a shared mainline several times a day. [1] That certainly does help with avoiding conflicts. There are only so many conflicts that you can pick up in a few hours of work. [1] https://en.wikipedia.org/wiki/Continuous_integration https://en.wikipedia.org/wiki/Continuous_integration

anon23anon 4y ago

Feel like the conversation went something like this - - We need this thing done by <insert date> -- That's impossible - What if we hire more ppl? -- They'd just step over each other - Let's divide things up then -- ok * six months pass by * - why isn't it done yet? -- we had to onboard and train all those new hires which sucked up a lot of dev time plus the new architecture added complexity and increased communication costs - How can we make our onboarding process faster? * sigh and the cycles repeats it self

aloisdg 4y ago
"What one programmer can do in one month, two programmers can do in two months." Fred Brooks

chronofar 4y ago

> Microservices was always a solution to the organisational problem of getting more developers working on a system at once. I’m not sure that that’s entirely true. To be sure that’s an aspect, but it’s also conceptually quite reasonable to desire the best tool for a particular job rather than employ a kitchen sink for everything. Separation of concerns is a commonly admired quality in code, and it makes sense sometimes to have this separation be at the repo level. I don’t jive with the “only organizational not technical” characterizations this thread is rife with. It’s now hipster to say the cool shiny thing is lame, and it blinds some utility. > It is easier to start with a monolith, find the right design and then split on the boundaries than it is to make the correct microservices to begin with. This I definitely agree with in many, perhaps most, cases. But of course inertia, especially in organizations, could have future you lamenting such a decision you now have no easy way out of.

hintymad 4y ago

Not sure if this story is true: Okta used to have a monolith architecture on AWS. All their 300 engineers used to work on the same service, so they had to run all their tests for every commit. The cost? $60 per commit.

cellis 4y ago
Sounds pretty cheap, relative to 300 engineers.
- samwillis 4y ago
  Assuming a dev is committing (ball park) 4 times per day, that's in the region of $55k/year per developer, or $16.5M/year all in. Thats not cheap...
  - djur 4y ago
    I would not assume that 300 developers are all committing 4 times a day. That seems very high.
    - jimbokun 4y ago
      And there you have it...$60 per commit means slowing down your commit velocity! Which means its longer before changes get tested or reviewed or deployed. The cost (money and more importantly time) of running tests fundamentally changes the development process.
      - xorcist 4y ago
        More commits are not better.
        JMS2021 4y ago
        Unless you are a PHP developer
  - 4y ago

staticassertion 4y ago

> Microservices was always a solution to the organisational problem of getting more developers working on a system at once. I don't really agree. It's probably the most agreed upon benefit, but there are others. Fundamentally, microservices allow you to isolate state. I can take two services and split them up - and now I've split their state spaces. I can put a queue service between them and now I've sliced out the state of their communication. This slicing up of state has a ton of benefits if done right. 1. Isolation of state is basically the key to having scalable concurrency. Watch the Whatsapp talk on Erlang and they say "Isolation, Isolation, Isolation". 2. Isolation of services is great for security. You can split up permissions across your services, limit their access in a more granular way, etc. Those are pretty nice wins. They're achievable through discipline in a monolith (heavy use of module boundaries, heavy use of immutability - something most mainstream languages don't encourage) but a network boundary really forces these things and makes unintentional stateful coupling a lot more painful. > It is easier to start with a monolith, find the right design and then split on the boundaries than it is to make the correct microservices to begin with. I disagree. Starting with a bad set of microservices can be fixed by merging. Merging two codebases is trivial compared to splitting. Again, if I have isolation between the services, even if the slicing up was done badly, even if they're coupled, I can just remove the layer between them. Splitting has to start from a place of coupling and then try to decouple - this is especially hard with languages that encourage encapsulated mutable state (most of them).

manigandham 4y ago
Network boundaries cause far more problems than they solve, and you've just shifted the complexity to now securing the network, usually with even more additional services, proxies, services meshes, firewalls, etc.
- staticassertion 4y ago
  > Network boundaries cause far more problems than they solve, They cause exactly 0 extra problems. A call from function A to function B can fail due to B having a bug. A call from service A to service B can fail due to B having a bug or a network failure. Either way, failure is possible and has to be handled - the network only makes that more obvious. Further, a call between functions can cause mutated shared state - not the case across a boundary, they physically do not share mutable state. > and you've just shifted the complexity to now securing the network Not really. Fundamentally you have split your service capabilites up - now you can apply least privilege as you desire.

dec0dedab0de 4y ago

Microservices was always a solution to the organisational problem of getting more developers working on a system at once. It may be a solution, but there were already solutions for that. As far as I remember, the orignal argument of microservices was to allow different teams to use whatever languages/tools they choose, as long as they provided an API. As opposed to having IT, payroll, and sales all having to wait for one development team to make everything. Then with docker and containers it became that you could isolate deployments, and scale out just the parts that need it. Which is great if you need to minimize outages, and have some parts of your system that get major spikes in traffic, and you decide it's worth the extra complexity.

blantonl 4y ago
This is where micro-services come in for me. Except, for all intents and purposes I'm a one man development shop. APIs. I've deployed about 10 or micro-services for use cases where a simple API is required for my applications. Maybe it's literally querying a row in a table and returning the result in JSON format. Or maybe it's to generate a JWT to access Apple Maps. Just craft it up in NodeJS, deploy to Lambda, and those little services haven't changed in years but they just keep cranking away. We can have both monoliths and micro-services where appropriate.
- deleted 4y ago
  [deleted]
- mason55 4y ago
  At the risk of committing a "No True Scotsman" fallacy, what you're talking about is not really what people are talking about when they are discussing the tradeoffs of a microservices architecture. While "microservice" obviously literally means a small service, when people talk about microservice architecture they're usually talking about building out applications that look like a single service to the end-user but in reality are a whole bunch of small services that talk to each other. Just creating a small service that exposes a small piece of functionality is clearly the right thing to do in many cases, but that's not what the discussion about microservices architectures is really about.

matchagaucho 4y ago

But also, according to Conway's Law, you end up with a product with leaky abstractions that reveal the underlying micro-services and org structure.

jbluepolarbear 4y ago

A better solution would be to build modules independently and have a single service use each module. Then things are unified, each team has a client it must deliver to, and issues must be fixed in order for them to be released in the final product. How are microservices better? Often each service is out of sync with each other, one service will upgrade and break another, that’s not possible with a single source and multiple modules.

pixl97 4y ago
Do people not version their API requests?
- jbluepolarbear 4y ago
  Yep and it gets worse, you’ll have teams add new breaking features or deprecate features without letting the other team know. Hey their tests pass so what do they care. I’ve worked with both strategies and modules over microservices are easier to integrate, ensure integrity, and reduces the probability that stuff breaks in production. Often microservices develop to a set of requirements and never consider the client that will be using that service. With modules that can’t happen.

strictfp 4y ago

Organizing around software components is such an idiotic idea. Good luck changing your design or making end-to-end features when you have teams protecting "their" part of the software. You've just made sure that any significant change to the system requires coordinating many different teams, making it at least one order more difficult than before, and a lot more bureaucratic.

peterangular 4y ago
For my anecdotal experience, microservices is the introduction of fiefdoms in development. And, I get it. Selfishly I would love to come onto a project and only have to worry about net-new development. It is hard to learn, and then positively contribute within another engineered system... of any complexity. When it's "my code" I tend to be much more productive, and it's easier to find my way through the forest. Anymore, everyone is on a power-grab for "my tools" and "my way" in regards to everything they're working on with near-zero attempt to understand the base needs of the business/feature. Microservices fill this need well as devs can work with their preferred toolset and just setup I/O boundaries for everyone else to use their service. Also, this is good for resume-driven development as you can often claim creation/ownership over some "piece" which looks way better on paper vs. "contributed to a big 'ol project." I'm not against microservices, I'm just being honest with how I've seen them socially play out - all of this is anecdotal to my experiences.
- dragonwriter 4y ago
  > For my anecdotal experience, microservices is the introduction of fiefdoms in development. Fiefdoms, whether implicit or explicit, are pervasive in organizations beyond trivial size; IME, software architecture choices may impact the shape of the fiefdoms, but not their existence.
deleted 4y ago
[deleted]

billiam 4y ago

>It is easier to start with a monolith, find the right design and then split on the boundaries than it is to make the correct microservices to begin with. Not pleasant, but true.

papito 4y ago

Dropbox - monolith Instagram - monolith StackOverflow - monolith (at least until SO Enterprise) There are many others. There is a reason why we avoided "distributed systems" in the past. It's insanely hard. Microsevices solve some problems in theory but create actual problems in practice. For most companies out there, distributed systems solve problems the companies don't even have. But, boy, do they need 10x more engineers to maintain this all.

hakre 4y ago
An they pose(d) some attraction. A boss after getting tired developing the baby that turned into a monolith and brought enough wealth to the table started to outline a new system with 12 or so microservices. Each one could be designed and planned and all written out in nice 2 to 6 page documents - one for each service. Now guess the size of the code wrangler team. And as this was so super-extravagant, guess the number of developers who were "promoted" to work on those micro-services. > But, boy, do they need 10x more engineers to maintain this all. Well stupidity saves you here. Ignore the one team per microservice rule and swap it with the "one-size-fits-it-all" developer rule: just promote one of the single team to be responsible for all micro-services. One lunatic that pushes code fast enough that management does not realize what is going on. Once everything is up and running, it just runs. The rest is downtime eventually, but you know what, its a software problem, what could be done? As long as everyone involved looks busy - and it's easy with plenty of micro-services to play whack-a-mole, bring one up again, kill two others, it is just a fail in the design as they should have been designed with failover but that was missed!
bin_bash 4y ago
Dropbox migrated to Go microservices
- papito 4y ago
  Fair enough. At that scale it's different. My point is that many major companies were a productive team working with a monolith until it simply was not feasible.

ChrisCinelli 4y ago

Microservices are not to scale organization even if some organizations used them in this way. There is a big problem with this way of scaling. The sense ownership becomes the sense of defend your castle (team's microservice code) from invasion (external developer that are trying to add features or fix bugs). Microservices were also presented as a solution for encapsulation. But if you cannot write good encapsulated code in a monolith, what makes you think microservices are the answer? http://www.codingthearchitecture.com/2014/07/06/distributed_big_balls_of_mud.html http://www.codingthearchitecture.com/2014/07/06/distributed_... Microservices (or just services) are the solution of having more bandwidth for a set of features without having to more of everything else and preventing some part of the code to influence negatively some others.

WalterBright 4y ago

Analogy is in programming itself - divide up a program into modules that each have a very clearly defined interface. Then people can work in parallel on it. I know everybody agrees with this, and it is stupidly obvious. But time after time, I see source file after source file importing (or #include'ing) every other module in the project. A pull request to fix a problem affects 10 files instead of one. It seems to me that people are simply terrible at modularizing code. (Including me.) I've seen this problem far more often than not, in every organization I've worked with, for my entire career. Just the other day I worked on a function that did some Windows API calls for file I/O, allocated a bunch of memory, and processed the file contents. I split it up so the code that needed the Windows API did just that - it didn't allocate any memory, and the processing was done with a callback. That meant the Windows API accessing code didn't need to import anything from the rest of the program, and the rest didn't need to import the Windows API. (There are some success stories. The popularization of generic algorithms and ranges (or iterators) has been a big improvement. And the use of global variables has become frowned upon.)

h2odragon 4y ago
> people are simply terrible at modularizing code. Once you realize you needed it, its too late. Then once you've been burnt on that fire for a bit, you go all out in designing the next few projects with framework and forethought and wind up with 100 lines of that for a 10 line script. the moral is "life's a bitch"
- 1penny42cents 4y ago
  That oscillation is necessary to be able to converge on the optimal medium for any given problem. Otherwise we get stuck in "modularization bad" or "modularization good" mindsets, where both are wrong.
dkarl 4y ago
I agree; people are much more careful about splitting responsibilities and designing APIs between processes (REST, GraphQL, XML-RPC) than between code modules. Simply switching from a monolith to microservices makes them more thoughtful about the consequences of design.

dragonwriter 4y ago

> Microservices was always a solution to the organisational problem of getting more developers working on a system at once. Microservices was always a solution to the problem that SOA got swallowed by “implementing XML heavy standards” and lost attention to it's architectural principles. (SOA was originally, and microservices a return to, a solution to reusing components and isolating faults so you need fewer, not more, developers and support personnel dedicated to any given system or collection of systems.)

outworlder 4y ago

> Microservices was always a solution to the organisational problem of getting more developers working on a system at once In other words, it's a great way to ship your org chart. I've only ever seen a handful of examples where the microservice boundaries were set regardless on how the actual teams are organized in the reporting structure.

icambron 4y ago
You'll ship your org chart whether you try to or not. There are two questions: a) will you architect your software to make this process (relatively) efficient? and b) to what degree are you willing to change your org chat to improve your architectural boundaries?
- mwcampbell 4y ago
  I think one can control the degree to which software architecture reflects the org chart. To take an extreme example, Chrome is primarily a single, giant (~200 MB) binary. I'm sure the layout of the source repositories reflects the Chrome org chart. But thankfully, they always had the engineering discipline to produce the architecture that was best for performance (a monolith, albeit a multi-process one) without letting it get out of control.

logifail 4y ago

> It is easier to start with a monolith, find the right design and then split on the boundaries [..] This reminds me of the discussion on let pedestrians define the walkways[0] [1] "As time goes on, we get smarter. We learn more about ourselves or our customers — what we or they really want. Therefore, we’re at our dumbest at the beginning, and at our smartest at the end. So when should you make decisions? When you have the most information, when you’re at your smartest: as late as possible." [0] https://sive.rs/walkways https://sive.rs/walkways [1] https://www.theguardian.com/cities/2018/oct/05/desire-paths-the-illicit-trails-that-defy-the-urban-planners https://www.theguardian.com/cities/2018/oct/05/desire-paths-...

pbreit 4y ago

Where I've found "microservices" to be sensible is where you have multiple different clients to power (ie, website, iOS & Android). They all consumer the same JSON and render natively. Otherwise, microservices just turn what was once easy into very complicated.

lamontcg 4y ago

A technology reason to have services is that you can have large RAM/SSD caches on services and move that data off of your front end monoliths, shrinking the footprint of the monolith and keeping more data accessible quickly (although small caches for the very hottest data might be good to retain on the first tier). That data probably shouldn't be peeled off though until there's a software team there to maintain the new service, and it should really be living in its own data pool on the backend (you shouldn't be writing a service to front a single database table, you should be writing a service to front an entire database). If the data is large enough and important enough to be moved to a service, and there's performance wins by doing so, and if there's a team that can manage that service, then it probably is a good idea. A single team that is managing a monolith busting it up into a dozen services that they still manage, and all the data comes out of a single database server is probably the wrong way to do it. There might be some other cases where you have hard business requirements and such for some software to be able to ship without other software and need to build clean lines and services so features can be extracted and probably some other edge cases, but the larger point should be that extracting the service should almost be demanded, both technologically and organizationally. If you have a choice and it isn't clear, then a service is probably the wrong thing. Services also really shouldn't be "micro" until you're an absolutely huge company and you can burn entire teams on small edge cases.

jacobsenscott 4y ago

> A lot of people working on one code base over the top of each other often causes problems and issues with ownership. Microservices don't fix this. If I'm working on a feature in a monolith and I need to modify code someone else is also working on for some other feature that same situation would occur in a microservices based app. Just because the code is in a different git repo, and called with via an RPC doesn't change the fact that you need to change it. Most software is monolithic, and most critical software is millions lines of code that hundreds or thousands of people have worked on - all without using microservices.

deleted 4y ago

[deleted]

mbrodersen 4y ago

Linux is a monolith. With more than 25 developers I believe. Seems to be doing OK last time I looked.

mbrodersen 4y ago

There is this amazing technology called “libraries”, “modules”, and “frameworks” that allows thousands of developers to work on the same monolith. It has been around for quite a while. No micro-services required. There is even this amazing technology called “automatic testing” that can test each library/framework independently before delivering to clients. I happens to use both of those amazing technologies at work every day and it works really well. I also happen to be sharing an office with a college who is fighting a 20 years old micro-services application. The worst piece of garbage I have ever seen in my 25+ years career. Every little functionality is running in its own process. There are hundreds of them. The complexity and difficulty of the system is truly mind blowing. If you think monoliths are hard, just wait until you split the same functionality into a distributed collection of micro-services. Welcome to the world of distributed systems. The additional failure scenarios just sky rocketed. Have a nice day!