5 ms·

Domain-Oriented Microservice Architecture (2020)

69 points by yla92 3y ago

xwowsersx 3y ago
> A single regression within a monolithic code base can bring the whole system (in this case, all of Uber) down. Serious question: why though? I can't think of a backend system where a single regression could bring the whole thing down. If something blows it up, it rarely takes the whole application down. What's this scenario exactly? Maybe I'm only thinking of certain types of applications, but a single failure somewhere needn't bring the whole thing down. > Poor separation of concerns. It was difficult to maintain good separations of concerns with a huge code base. In an exponential growth environment, expediency sometimes led to poor boundaries between logic and components. I kind of get this...The idea is that if the other area of code simply isn't accessible to you, you're forced to keep the concerns separate? Seems like this could be dealt with by applying more discipline and just making sure to design things to be more modular and.. separate. This is the hallmark of any good code base. Do we need an entirely different architecture, with all the pitfalls that microservices entail, because we can't seem to keep things clean and separate? I really am asking these questions in good faith. I won't hide the fact that I'm skeptical of the microservices fashion (and I know that Uber is at least trying to be more thoughtful here), but I am totally open to being convinced otherwise. I'm at this very moment gearing up to begin a new large project and am wondering what design makes the most sense and whether there's a place for separate services so I'm curious to hear what people think. The one area I'm thinking about specifically is the fact that difference parts of a monolith might have different scaling requirements and if you have a single application then everything has to scale out or in as a unit. But it seems like this doesn't require a separate code base necessarily...just the ability to run different parts of a single code base as its own applications or unit of deployment, no? This old post from Michael DeHaan comes to mind as I'm thinking about this https://archive.ph/YMRYx https://archive.ph/YMRYx
- nivertech 3y ago
  > A single regression within a monolithic code base can bring the whole system (in this case, all of Uber) down. Serious question: why though? I can't think of a backend system where a single regression could bring the whole thing down. If something blows it up, it rarely takes the whole application down. What's this scenario exactly? Maybe I'm only thinking of certain types of applications, but a single failure somewhere needn't bring the whole thing down. A crash in a stateless distributed tier (e.g. web servers) will not take the whole system down. But a crash while processing any request in a stateful non-distributed multithreaded system, can bring it down. It's much harder to make stateful systems resilient and fault tolerant. Usually it's solved by moving the state somewhere else (e.g. databases, message queues, or Kafka-style AOLs). Exception: monolithic stateful systems using BEAM/OTP/Erlang/Elixir. Here the blast radius is per Erlang process, so in the worst case only a single request/user/session should be affected. There are some classes of errors (e.g. out-of-memory, overloaded process message queues, or crash in native code library) which can bring down an entire Erlang node. This can be mitigated by running a cluster of several Erlang nodes (which is supported out-of-the box by BEAM VM and OTP).
  - CharlieYuuu 3y ago
    I was experience a case : whole system (a stateless distributed tier) down caused by a user input (the user try again and again until all instances down).
    - nivertech 3y ago
      Crashed instances should restart. On EC2 one can put all instances running stateless code into autosacling ground behind load balancer, or ReplicaSet for deployment/service in k8s. BTW: I expirienced the entier us-east region going down just in time when I was demoing my product to a VC. Turns out it was a bug in an AWS Management Console for that region.
  - knewter 3y ago
    Erlang/elixir is a very nice middle ground situation where you can build "monolithically" but also inherently be building services that are isolated, and you can scale to 50-150 nodes before getting any more complicated. 150 nodes is a lot. I love the beam.
    - nivertech 3y ago
      That's for the standard full mesh clustering, Erlang distribution can be further tuned with hidden nodes, etc. I think WhatsApp scaled clusters to 10_000 nodes[1]. I personally only run max 10 nodes Erlang cluster (on physical servers) in production, and ~ 20 on AWS for dev/testing workloads. But I think it's better to have a federation of smaller clusters, or separate cluster per datacenter/AZ. Erlang distribution isn't a silver bullet, though. But for 80% of cases it's enough. --- [1] Scaling Erlang cluster to 10,000 nodes https://codesync.global/media/maxim-fedorov-scaling-erlang-cluster-to-10-000-nodes/ https://codesync.global/media/maxim-fedorov-scaling-erlang-c...
      - knewter 3y ago
        Yeah then also there's partisan. I didn't figure anyone that needed to go beyond 150 nodes would fail to learn this stuff, but I appreciate you mentioning it so I don't accidentally drive-by convince someone there's an upper limit rather than engineering tradeoffs :)
- mjr00 3y ago
  > The idea is that if the other area of code simply isn't accessible to you, you're forced to keep the concerns separate? Seems like this could be dealt with by applying more discipline and just making sure to design things to be more modular and.. separate. This is the hallmark of any good code base. Do we need an entirely different architecture, with all the pitfalls that microservices entail, because we can't seem to keep things clean and separate? Yes, because code is written by humans, and humans make mistakes. "Just have discipline and write good code" doesn't scale, especially in, as Uber's case, quickly growing organizations where large numbers of people new to the codebase are contributing.
  - charcircuit 3y ago
    What is it about network calls that makes it easier to scale separation over function calls. To me the RPC you choose should be independent from the separation part of it.
- localghost3000 3y ago
  > I'm at this very moment gearing up to begin a new large project and am wondering what design makes the most sense IMO a monolith is always the starting point until your team gets big enough to justify breaking it up. > Seems like this could be dealt with by applying more discipline and just making sure to design things to be more modular and.. separate This really depends on the team size. As you get above, say, 30 engineers contributing to your codebase daily communicating those logical boundaries becomes nigh impossible without significant operational overhead. You either accept the chaos and let your codebase become a Big Ball Of Mud (most common) or you introduce so much process overhead that your team paradoxically gets slower the more people you add. I'm sure there are exceptions but it's rare. > Do we need an entirely different architecture, with all the pitfalls that microservices entail, because we can't seem to keep things clean and separate? Microservices are probably not the tool to reach for no. Several "fat" services separated along some kind of bounded context within your domain? I've seen that work great as a next step in a monolithic architecture evolution. Teams can have the autonomy they need to make decisions quickly, separation of concerns is enforced by the network. The extra overhead incurred isn't untenable since you only have a few services instead of a hundred.
  - deleted 3y ago
    [deleted]
- pydry 3y ago
  >I kind of get this...The idea is that if the other area of code simply isn't accessible to you, you're forced to keep the concerns separate? IME about half of the microservice architectures Ive worked on have managed to make sure that the concerns and network boundaries cut across completely different places. At least this mistake in a "monolith" (hate that word) is easier to undo.
  - jeltz 3y ago
    Yeah, in all micro service architectures I have worked in people have taken shortcuts which ended up in concerns spread all over different services. This misstanke can be done there just as well as in monoliths.
- KronisLV 3y ago
  > Serious question: why though? I can't think of a backend system where a single regression could bring the whole thing down. If something blows it up, it rarely takes the whole application down. What's this scenario exactly? Maybe I'm only thinking of certain types of applications, but a single failure somewhere needn't bring the whole thing down. Suppose you have a web based CRUD that processes HTTP requests. To do this, it connects to a DB. There is one thread pool for HTTP requests, there's another thread pool for DB connections. Suppose there's an issue that causes a DB connection to hang (either a really slow query, or an issue with the DB thread pooling solution itself). All of the sudden, the DB thread pool is slowly getting exhausted, until no more queries can run because everything is hanging. Soon after, the HTTP request thread pool fills up, because requests are waiting for processing but none is being done. Users either see their requests frozen, or, if you're packaging both the API and the front end resources in the same deployment unit, soon nobody can even load the static resources of the page. I've seen this happen with a Java web app that used an old DB pooling solution that caused a prod outage years ago. Had to rip it out and replace it with something else. That said, the same could happen with stuff like scheduled processes conflicting, memory exhaustion, CPU utilization, disk usage and so on. Now, there shouldn't be anything preventing you from running multiple instances of monolithic codebases in parallel, though sometimes systems aren't written with that in mind (e.g. no DB queues/RabbitMQ for messaging/scheduled processes, no assumption of multiple instances accessing the same data, in-memory caches instead of Redis and so on) and you end up with a singleton application instead, which is worse than just a monolith. Either way, planning for components/systems failing and being able to recover from that is almost always a good idea (health checks, automated restarts, some horizontal scaling, uptime notifications and alerting, some instrumentation like APM), unless you're really short on time. None of that means that you should start with microservices though, unless your whole team is really comfortable with that. Monoliths are okay, as are modular monoliths (same codebase, deploy different features for app instances with feature flags, like FEATURE_ENABLE_API, FEATURE_ENABLE_SCHEDULED_PROCESSES etc.).
- asimpletune 3y ago
  I'm also generally micro-service skeptical, but I'll do my best to give what I believe an answer is. The industry had a huge micro service phase for like the last decade, and I think in the end most of the same arguments given for micro services could be answered with just modules. Someone else can correct me where I'm wrong, but I think maybe the one argument for a separate service vs module is security, where you want literal process isolation for data. I guess another benefit is micro services make it easier for teams to work in whatever language/tool they're comfortable with. I think that's it really. Better defined ownership could be done with modules. Testability could be done with modules. Most of the usual arguments could just be answered basically with well defined code, and when the interfaces are all async then it even means even those modules could have micro services as their underlying implementation.
  - asimpletune 3y ago
    As a side note, I think even a lot of the data/process isolation stuff could be better mitigated by capabilities-based technology. It's kind of crazy to me that stuff isn't used more.
  - DrScientist 3y ago
    There are two aspects here I think. Modularity in terms of code, and modularity in terms of deployment lifecycle. As you say most dependency problems can be handled at the level of the code, depending on how you structure the builds etc. ( personally I think automatically pulling in dependencies is a mistake - it should be slightly painful to add a new dependency ). For me, the reason for microservice like deployment is having a separately lifecycle - being able to start,stop, update separately from other services. Another aspect which I don't think is talked about enough, is that every deployment has a copy of the code ( in whatever executable form ) - this means I can deploy version 1 of code A in service X and version 2 in service Y at the same time if I need to - ie I can incrementally update the system. So one way to think about what should be deployed together ( or not ) is thinking not about the code per se - but also the natural deployment cycle.
  - movpasd 3y ago
    It's not just about code -- microservices can offer a degree of modularity on an infrastructure level. For example, a microservice can own its own database. Being able to experiment with languages and tools (as you mention) is also a pretty big advantage. HTTP offers a universal communications interface beyond language-level APIs and ABIs. When we think about an entire department with many teams with a diversity of expertise, at least we can guarantee everyone understands HTTP.
- j0057 3y ago
  An async deadlock in a buggy library will under load exhaust the process threads and take down everything. One bug can cause large objects to be allocated, freaking out the garbage collector, causing increasingly large lag spikes under load, and timeouts in seemingly unrelated code paths. A configuration change affecting only one part of the system can prevent the entire process from starting up. Modularization is one valid way to increase resilience by limiting blast radius.
  - gpderetta 3y ago
    You can fork(2) in monolithic applications if you want address space separation.
- Sakos 3y ago
  I'm not sure why it's suddenly so difficult to take down a monolithic application. It's totally doable and isn't that difficult. It happens all the time too. For me, the funny thing is I've found that it can be just as easy for a single broken microservice to bring down everything else, and it can be a huge pain figuring out where such a problem originated from because the wrong services are erroring out.
- ManuelKiessling 3y ago
  Throwing in the Self-Contained Systems approach for your consideration. Not necessarily the One-Architecture-To-Rule-Them-All (none is), but as you've mentioned a "new LARGE project", it might be a good choice. See https://scs-architecture.org https://scs-architecture.org and https://manuel-kiessling-net.translate.goog/2016/01/02/die-architektur-der-galeria-de-plattform-im-kontext-der-produktentwicklungsorganisation/?_x_tr_sl=de&_x_tr_tl=en&_x_tr_hl=de&_x_tr_pto=wapp https://manuel-kiessling-net.translate.goog/2016/01/02/die-a... for more information.
  - xwowsersx 3y ago
    This is great, thanks very much for sharing
netfortius 3y ago
Stack Overflow doesn't seem to have this issue.
- m0llusk 3y ago
  Stack Overflow is missing some of the key stuff that generates complexity in the Uber systems. Calculating the value of the service offered and pairing questioners with responders might potentially be really useful additions to Stack Overflow. And instead of physical maps services could be plotted on maps of languages, frameworks, interfaces, and other software abstractions.
- dilyevsky 3y ago
  SO is read-heavy/write-almost-never system
charcircuit 3y ago
>Availability Risks. A single regression within a monolithic code base can bring the whole system (in this case, all of Uber) down. A single regression in your isEven microservice can also take the whole system down. You can mitigate this by slowly rolling out new versions to try and catch regressions early so you can revert. >Risky, expensive deployments. These were painful and time consuming to perform with the frequent need for rollbacks. Code that is going to be deployed all needs to be deployed. Painful deploys are going to happen with microservices too. >Poor separation of concerns. It was difficult to maintain good separations of concerns with a huge code base. In an exponential growth environment, expediency sometimes led to poor boundaries between logic and components. Making libraries will boundaries without having to do an expensive network call. >Inefficient execution. These issues combined made it difficult for teams to execute autonomously or independently. Monoliths let you work independently too. You go in and change something and now it is changed. No part of monoliths make it harder as a single person or team to make changes. Also keep in mind Uber ships Android and iPhone apps which are monoliths. They don't force you to install an app for entering a location, an app for making a payment, an app for looking up pricing, etc.
- tsss 3y ago
  > isEven microservice If you have one then you're doing it wrong and don't get the most basic thing about microservice architecture. Microservices must be business-aligned and as independent as possible. > They don't force you to install an app for entering a location, an app for making a payment, an app for looking up pricing They probably do. I bet their Android app has independent dynamic feature modules that are loaded on demand.
  - charcircuit 3y ago
    >If you have one then you're doing it wrong No. Microservices as the name suggests are micro and their functionality can be pretty limited. If you don't have any trivial microservices then you are doing it wrong. You may be objecting because of how inefficient it seems to put a function like that over a network call, but that just makes the performance downside of the microservice architecture clearer.

69 points by yla92 3y ago

> A single regression within a monolithic code base can bring the whole system (in this case, all of Uber) down. Serious question: why though? I can't think of a backend system where a single regression could bring the whole thing down. If something blows it up, it rarely takes the whole application down. What's this scenario exactly? Maybe I'm only thinking of certain types of applications, but a single failure somewhere needn't bring the whole thing down. > Poor separation of concerns. It was difficult to maintain good separations of concerns with a huge code base. In an exponential growth environment, expediency sometimes led to poor boundaries between logic and components. I kind of get this...The idea is that if the other area of code simply isn't accessible to you, you're forced to keep the concerns separate? Seems like this could be dealt with by applying more discipline and just making sure to design things to be more modular and.. separate. This is the hallmark of any good code base. Do we need an entirely different architecture, with all the pitfalls that microservices entail, because we can't seem to keep things clean and separate? I really am asking these questions in good faith. I won't hide the fact that I'm skeptical of the microservices fashion (and I know that Uber is at least trying to be more thoughtful here), but I am totally open to being convinced otherwise. I'm at this very moment gearing up to begin a new large project and am wondering what design makes the most sense and whether there's a place for separate services so I'm curious to hear what people think. The one area I'm thinking about specifically is the fact that difference parts of a monolith might have different scaling requirements and if you have a single application then everything has to scale out or in as a unit. But it seems like this doesn't require a separate code base necessarily...just the ability to run different parts of a single code base as its own applications or unit of deployment, no? This old post from Michael DeHaan comes to mind as I'm thinking about this https://archive.ph/YMRYx https://archive.ph/YMRYx

nivertech 3y ago
> A single regression within a monolithic code base can bring the whole system (in this case, all of Uber) down. Serious question: why though? I can't think of a backend system where a single regression could bring the whole thing down. If something blows it up, it rarely takes the whole application down. What's this scenario exactly? Maybe I'm only thinking of certain types of applications, but a single failure somewhere needn't bring the whole thing down. A crash in a stateless distributed tier (e.g. web servers) will not take the whole system down. But a crash while processing any request in a stateful non-distributed multithreaded system, can bring it down. It's much harder to make stateful systems resilient and fault tolerant. Usually it's solved by moving the state somewhere else (e.g. databases, message queues, or Kafka-style AOLs). Exception: monolithic stateful systems using BEAM/OTP/Erlang/Elixir. Here the blast radius is per Erlang process, so in the worst case only a single request/user/session should be affected. There are some classes of errors (e.g. out-of-memory, overloaded process message queues, or crash in native code library) which can bring down an entire Erlang node. This can be mitigated by running a cluster of several Erlang nodes (which is supported out-of-the box by BEAM VM and OTP).
- CharlieYuuu 3y ago
  I was experience a case : whole system (a stateless distributed tier) down caused by a user input (the user try again and again until all instances down).
  - nivertech 3y ago
    Crashed instances should restart. On EC2 one can put all instances running stateless code into autosacling ground behind load balancer, or ReplicaSet for deployment/service in k8s. BTW: I expirienced the entier us-east region going down just in time when I was demoing my product to a VC. Turns out it was a bug in an AWS Management Console for that region.
- knewter 3y ago
  Erlang/elixir is a very nice middle ground situation where you can build "monolithically" but also inherently be building services that are isolated, and you can scale to 50-150 nodes before getting any more complicated. 150 nodes is a lot. I love the beam.
  - nivertech 3y ago
    That's for the standard full mesh clustering, Erlang distribution can be further tuned with hidden nodes, etc. I think WhatsApp scaled clusters to 10_000 nodes[1]. I personally only run max 10 nodes Erlang cluster (on physical servers) in production, and ~ 20 on AWS for dev/testing workloads. But I think it's better to have a federation of smaller clusters, or separate cluster per datacenter/AZ. Erlang distribution isn't a silver bullet, though. But for 80% of cases it's enough. --- [1] Scaling Erlang cluster to 10,000 nodes https://codesync.global/media/maxim-fedorov-scaling-erlang-cluster-to-10-000-nodes/ https://codesync.global/media/maxim-fedorov-scaling-erlang-c...
    - knewter 3y ago
      Yeah then also there's partisan. I didn't figure anyone that needed to go beyond 150 nodes would fail to learn this stuff, but I appreciate you mentioning it so I don't accidentally drive-by convince someone there's an upper limit rather than engineering tradeoffs :)
mjr00 3y ago
> The idea is that if the other area of code simply isn't accessible to you, you're forced to keep the concerns separate? Seems like this could be dealt with by applying more discipline and just making sure to design things to be more modular and.. separate. This is the hallmark of any good code base. Do we need an entirely different architecture, with all the pitfalls that microservices entail, because we can't seem to keep things clean and separate? Yes, because code is written by humans, and humans make mistakes. "Just have discipline and write good code" doesn't scale, especially in, as Uber's case, quickly growing organizations where large numbers of people new to the codebase are contributing.
- charcircuit 3y ago
  What is it about network calls that makes it easier to scale separation over function calls. To me the RPC you choose should be independent from the separation part of it.
localghost3000 3y ago
> I'm at this very moment gearing up to begin a new large project and am wondering what design makes the most sense IMO a monolith is always the starting point until your team gets big enough to justify breaking it up. > Seems like this could be dealt with by applying more discipline and just making sure to design things to be more modular and.. separate This really depends on the team size. As you get above, say, 30 engineers contributing to your codebase daily communicating those logical boundaries becomes nigh impossible without significant operational overhead. You either accept the chaos and let your codebase become a Big Ball Of Mud (most common) or you introduce so much process overhead that your team paradoxically gets slower the more people you add. I'm sure there are exceptions but it's rare. > Do we need an entirely different architecture, with all the pitfalls that microservices entail, because we can't seem to keep things clean and separate? Microservices are probably not the tool to reach for no. Several "fat" services separated along some kind of bounded context within your domain? I've seen that work great as a next step in a monolithic architecture evolution. Teams can have the autonomy they need to make decisions quickly, separation of concerns is enforced by the network. The extra overhead incurred isn't untenable since you only have a few services instead of a hundred.
- deleted 3y ago
  [deleted]
pydry 3y ago
>I kind of get this...The idea is that if the other area of code simply isn't accessible to you, you're forced to keep the concerns separate? IME about half of the microservice architectures Ive worked on have managed to make sure that the concerns and network boundaries cut across completely different places. At least this mistake in a "monolith" (hate that word) is easier to undo.
- jeltz 3y ago
  Yeah, in all micro service architectures I have worked in people have taken shortcuts which ended up in concerns spread all over different services. This misstanke can be done there just as well as in monoliths.
KronisLV 3y ago
> Serious question: why though? I can't think of a backend system where a single regression could bring the whole thing down. If something blows it up, it rarely takes the whole application down. What's this scenario exactly? Maybe I'm only thinking of certain types of applications, but a single failure somewhere needn't bring the whole thing down. Suppose you have a web based CRUD that processes HTTP requests. To do this, it connects to a DB. There is one thread pool for HTTP requests, there's another thread pool for DB connections. Suppose there's an issue that causes a DB connection to hang (either a really slow query, or an issue with the DB thread pooling solution itself). All of the sudden, the DB thread pool is slowly getting exhausted, until no more queries can run because everything is hanging. Soon after, the HTTP request thread pool fills up, because requests are waiting for processing but none is being done. Users either see their requests frozen, or, if you're packaging both the API and the front end resources in the same deployment unit, soon nobody can even load the static resources of the page. I've seen this happen with a Java web app that used an old DB pooling solution that caused a prod outage years ago. Had to rip it out and replace it with something else. That said, the same could happen with stuff like scheduled processes conflicting, memory exhaustion, CPU utilization, disk usage and so on. Now, there shouldn't be anything preventing you from running multiple instances of monolithic codebases in parallel, though sometimes systems aren't written with that in mind (e.g. no DB queues/RabbitMQ for messaging/scheduled processes, no assumption of multiple instances accessing the same data, in-memory caches instead of Redis and so on) and you end up with a singleton application instead, which is worse than just a monolith. Either way, planning for components/systems failing and being able to recover from that is almost always a good idea (health checks, automated restarts, some horizontal scaling, uptime notifications and alerting, some instrumentation like APM), unless you're really short on time. None of that means that you should start with microservices though, unless your whole team is really comfortable with that. Monoliths are okay, as are modular monoliths (same codebase, deploy different features for app instances with feature flags, like FEATURE_ENABLE_API, FEATURE_ENABLE_SCHEDULED_PROCESSES etc.).
asimpletune 3y ago
I'm also generally micro-service skeptical, but I'll do my best to give what I believe an answer is. The industry had a huge micro service phase for like the last decade, and I think in the end most of the same arguments given for micro services could be answered with just modules. Someone else can correct me where I'm wrong, but I think maybe the one argument for a separate service vs module is security, where you want literal process isolation for data. I guess another benefit is micro services make it easier for teams to work in whatever language/tool they're comfortable with. I think that's it really. Better defined ownership could be done with modules. Testability could be done with modules. Most of the usual arguments could just be answered basically with well defined code, and when the interfaces are all async then it even means even those modules could have micro services as their underlying implementation.
- asimpletune 3y ago
  As a side note, I think even a lot of the data/process isolation stuff could be better mitigated by capabilities-based technology. It's kind of crazy to me that stuff isn't used more.
- DrScientist 3y ago
  There are two aspects here I think. Modularity in terms of code, and modularity in terms of deployment lifecycle. As you say most dependency problems can be handled at the level of the code, depending on how you structure the builds etc. ( personally I think automatically pulling in dependencies is a mistake - it should be slightly painful to add a new dependency ). For me, the reason for microservice like deployment is having a separately lifecycle - being able to start,stop, update separately from other services. Another aspect which I don't think is talked about enough, is that every deployment has a copy of the code ( in whatever executable form ) - this means I can deploy version 1 of code A in service X and version 2 in service Y at the same time if I need to - ie I can incrementally update the system. So one way to think about what should be deployed together ( or not ) is thinking not about the code per se - but also the natural deployment cycle.
- movpasd 3y ago
  It's not just about code -- microservices can offer a degree of modularity on an infrastructure level. For example, a microservice can own its own database. Being able to experiment with languages and tools (as you mention) is also a pretty big advantage. HTTP offers a universal communications interface beyond language-level APIs and ABIs. When we think about an entire department with many teams with a diversity of expertise, at least we can guarantee everyone understands HTTP.
j0057 3y ago
An async deadlock in a buggy library will under load exhaust the process threads and take down everything. One bug can cause large objects to be allocated, freaking out the garbage collector, causing increasingly large lag spikes under load, and timeouts in seemingly unrelated code paths. A configuration change affecting only one part of the system can prevent the entire process from starting up. Modularization is one valid way to increase resilience by limiting blast radius.
- gpderetta 3y ago
  You can fork(2) in monolithic applications if you want address space separation.
Sakos 3y ago
I'm not sure why it's suddenly so difficult to take down a monolithic application. It's totally doable and isn't that difficult. It happens all the time too. For me, the funny thing is I've found that it can be just as easy for a single broken microservice to bring down everything else, and it can be a huge pain figuring out where such a problem originated from because the wrong services are erroring out.
ManuelKiessling 3y ago
Throwing in the Self-Contained Systems approach for your consideration. Not necessarily the One-Architecture-To-Rule-Them-All (none is), but as you've mentioned a "new LARGE project", it might be a good choice. See https://scs-architecture.org https://scs-architecture.org and https://manuel-kiessling-net.translate.goog/2016/01/02/die-architektur-der-galeria-de-plattform-im-kontext-der-produktentwicklungsorganisation/?_x_tr_sl=de&_x_tr_tl=en&_x_tr_hl=de&_x_tr_pto=wapp https://manuel-kiessling-net.translate.goog/2016/01/02/die-a... for more information.
- xwowsersx 3y ago
  This is great, thanks very much for sharing

netfortius 3y ago

Stack Overflow doesn't seem to have this issue.

m0llusk 3y ago
Stack Overflow is missing some of the key stuff that generates complexity in the Uber systems. Calculating the value of the service offered and pairing questioners with responders might potentially be really useful additions to Stack Overflow. And instead of physical maps services could be plotted on maps of languages, frameworks, interfaces, and other software abstractions.
dilyevsky 3y ago
SO is read-heavy/write-almost-never system

charcircuit 3y ago

>Availability Risks. A single regression within a monolithic code base can bring the whole system (in this case, all of Uber) down. A single regression in your isEven microservice can also take the whole system down. You can mitigate this by slowly rolling out new versions to try and catch regressions early so you can revert. >Risky, expensive deployments. These were painful and time consuming to perform with the frequent need for rollbacks. Code that is going to be deployed all needs to be deployed. Painful deploys are going to happen with microservices too. >Poor separation of concerns. It was difficult to maintain good separations of concerns with a huge code base. In an exponential growth environment, expediency sometimes led to poor boundaries between logic and components. Making libraries will boundaries without having to do an expensive network call. >Inefficient execution. These issues combined made it difficult for teams to execute autonomously or independently. Monoliths let you work independently too. You go in and change something and now it is changed. No part of monoliths make it harder as a single person or team to make changes. Also keep in mind Uber ships Android and iPhone apps which are monoliths. They don't force you to install an app for entering a location, an app for making a payment, an app for looking up pricing, etc.

tsss 3y ago
> isEven microservice If you have one then you're doing it wrong and don't get the most basic thing about microservice architecture. Microservices must be business-aligned and as independent as possible. > They don't force you to install an app for entering a location, an app for making a payment, an app for looking up pricing They probably do. I bet their Android app has independent dynamic feature modules that are loaded on demand.
- charcircuit 3y ago
  >If you have one then you're doing it wrong No. Microservices as the name suggests are micro and their functionality can be pretty limited. If you don't have any trivial microservices then you are doing it wrong. You may be objecting because of how inefficient it seems to put a function like that over a network call, but that just makes the performance downside of the microservice architecture clearer.