7 ms·

> Organizing your local software systems using separate processes, microservices that are combined using a REST architectural style, does help enforce module bo

> Organizing your local software systems using separate processes, microservices that are combined using a REST architectural style, does help enforce module boundaries via the operating system but at significant costs. It is a very heavy-handed approach for achieving modularity. One of the most effective and succinct criticisms I've seen of microservices: an architecture hack for modularity. The irony is, it's rarely an effective hack. If you don't get your service boundaries right, modularity goes out the window, and you're stuck with a bunch of network calls that should be function calls.

andrew_ 4y ago
Agreed. The kicker is knowing what those boundaries are and avoiding them. I really enjoy working with microservices for async tasks, and I think they're really well-suited to them. I really dislike inter-service dependencies. And of course, there's nothing wrong with a few, small monolith services. It's all about the right tool for the right job for me, and being too prescriptive to a particular architecture comes at a high cost.
ravenstine 4y ago
A better first approach would be to try and remove roadblock and bureaucracy that are encouraging developers to actually consider microservices. Those are of course not the only the only arguments for microservices, but I do think they're a big one that more often goes unsaid than the modularity. You can have modularity in a monolithic app, but for some reason many teams don't actually take modularity seriously; they ask that everything be object-oriented and call it a day, as if writing a `module` in Ruby means you've made things modular. Microservices become appealing when the process to have monolithic code reviewed and deployed is long and painful, since they are by definition small, decoupled, and simple to deploy. Of course it often doesn't quite work out that way, in which case a monolith would have worked anyway. But it's unlikely things will change in coding culture so long as there's the perverse incentive for businesses to encourage their engineers to use hacks out of expediency. Hacks inevitably make things really hard down the road, create mysterious problems that are hard to solve, and the solution is often to add bureaucracy or switch to microservice architecture.
- ar_lan 4y ago
  Yeah, I agree. I've worked with a variety of monoliths. The biggest complaint I've had from companies who utilize that architecture is that I shouldn't have to run entire pre and post pipelines to make a minor change to an internal function that only affects one package in the entire repository. This is solvable, but it is a problem - one that causes a lot of engineers to consider starting fresh in a new repo anyway. Multiple repositories inherently solve this for you - but bring about other problems. In my opinion - most projects don't need microservice architecture in the sense of separated binaries. Typically, they can be written with clear boundaries within the monolith such that, should the time come when scaling concerns are real for certain applications, then they can be ripped out as needed into their own binaries.
  - potbelly83 4y ago
    Interesting question arises here. Given most shops are now using interpreted languages, is it possible to make a change to say a .py file and ONLY deploy that .py file to production? i.e. do incremental releases
    - ar_lan 4y ago
      I don't know if "most shops are now using interpreted languages" is an accurate statement :) (I actually don't know). But I think that's definitely an interesting idea (likely with some small levels of extra caution with some specific high profile files, like an API definition/endpoint or something).
      - potbelly83 4y ago
        Ha no worries, to be fair most of my working life has been spent at C++ shops. I guess Windows does something similar via dlls.
    - pojzon 4y ago
      The issue was rather about the fact that changing that single .py file can have impact on a service A and service B which depend on service C where the file was changed. In microservice world, you have to run e2e tests because thats the only place where you can really guarantee the system will work as you expect. Pacts wont solve that, integration test - the same, units w/e. Running a monolith test is overall faster than spinning 20 microservices, beinging every db to specific atate and running tests.
blobbers 4y ago
While that's true that the network calls should be function calls, it means you can scale horizontally in a lot easier fashion. Sure you've got waste at the small scale, but if you're building for webscale then you have to realize these are the trade offs one must make.
- willcipriano 4y ago
  I've seen several times where the overhead of the network request is dramatically higher than the work being preformed. The worst example was a validation microservice that did things like "is True" or "is greater than 5" one element at a time on a large object. So you'd have an object with 50 elements and it would make 50 requests to the service. This was in the context of a batch processing job that handled millions of items, so billions of requests end up being created. I tried to explain that each network request probably does a few thousand things like "if request_type == 'get'" during it's lifecycle on both sides of the transaction, but nobody got it and I quit soon after.
  - brianwawok 4y ago
    A lot of devs just don’t care. They want to do a cool thing for their resume. This usually means there is no tech leadership that “gets it”.
    - runevault 4y ago
      Resume driven development is one of the most depressing things I keep running into in my career. Wanting to learn and grow is great, but forcing customers to suffer the repercussions of your learning because technology/design pattern x is in vogue is closing in on malpractice.
      - brianwawok 4y ago
        Until there is consequences, why not? Next time you interview with someone, ask why they converet a system to Microservices at their last job? If they don't have an actual reason, bin them.
ryanmcbride 4y ago
Yup I've seen it at just about every company I've worked for. A cluster of microservices that is really just a distributed monolith. Is it really a microservice if changing one part of it requires making echoing changes to every service downstream? Is it really a microservice if one part breaking causes the entire system to fail spectacularly? I feel like the pendulum is starting to swing the other way as companies learn that microservices aren't the end-all solution to their process problems, and that prioritizing new features every sprint and constantly pushing housekeeping, bugsquashing, and system improvements will result in a broken collection of microservices just as quickly as it resulted in a broken monolith.
- 411111111111111 4y ago
  > Is it really a microservice if one part breaking causes the entire system to fail spectacularly? No, that's a distributed monolith not a microservice
- cogman10 4y ago
  > prioritizing new features every sprint and constantly pushing housekeeping, bugsquashing, and system improvements will result in a broken collection of microservices just as quickly as it resulted in a broken monolith. Now, HTF do you convince a C-Level of this truth? Seems like there is never any time for any sort of housekeeping until the house is literally on fire and the whole world is looking at our smoke pillar. Even then, they'll usually be like "Ok, take 2 weeks to fix all our architecture problems and then get back to work on new features!". As SOON as the fire is doused, they want to move on rather than cleaning and removing the flammable material and the flame that keep starting the fire.
  - mixedCase 4y ago
    You don't, it's not his job to care, it's yours. And just like you don't ask permission for writing down a line of feature code, you don't ask permission for maintenance work. You just do it and schedule it accordingly to business needs to the best of your ability. And make sure not to apologize for doing your work or treat maintenance work as "unimportant" because that is how it will be perceived. It does get tricky when you have a manager defining tasks and you have yet to determine whether their job is to be a technical manager (and is adequately competent) or a nerd babysitter/translator. In the former case it's good to run maintenance projects by them if they need active, dedicated time as opposed to something you can mostly do during downtime. In case it's the other kind of "manager", and you've already tried and failed to convince them to prioritize basic stuff, you have to just put it inside other estimates and rope in the rest of your team to do the same and/or polish your résumé.
nine_k 4y ago
Why, SOA is a reasonable concept, and does help scaling. Say, billing, ETL and batch processing, and Web backends can live as separate services all right. Going with micro-services is another story, and it makes sense in a more narrow gamut of circumstances. Say, microservices may be a great fit for AWS Lambda-style deployments, but this assumes spiky, sparse load patterns.
- thibauts 4y ago
  You can deploy your whole monolith on one single lambda and it will scale the same. It may even help to keep it hot. Services, micro or not are only really useful when load factors are vastly different and you’re not serverless. Or when you have sufficiently many large teams and a sufficiently large codebase that a single deploy becomes unmanageable because too much communication / coordination is needed.
jasonwatkinspdx 4y ago
When the microservice craze took off a lot of people asked me for opinions on it, and said the same thing then I say now: it more or less boils down to Conway's law. If you have multiple teams that need to iterate and deploy independently the overhead may be worth it. But if you're just a small startup with a single unitary dev team, and that generally just deploys a new version of any changed services all as one lump, it's an insane amount of complexity and overhead.
revskill 4y ago
As long as microservices don't share database, then all is fine.
Srikanth 4y ago
This probably is the reason why it works in some settings. Anecdotally, as someone with more domain/functional knowledge and operations knowledge than knowledge of frameworks, I found microservices architecture with good functional test coverage a better way to deal with a team of programmers like me (basically, a typical team in an offshore IT consultancy building enterprise applications using SpringBoot and Node.JS). Ship the service as early as possible with available talent and then get someone really good with that programming language or framework to deal with performance bottlenecks within the microservice. I see it basically as a way to limit the blast radius of the applications. Of course, as you said, get the boundaries wrong and you have a bigger problem.
nonameiguess 4y ago
It kind of feels like Sid is lying through his teeth here, as a person who deploys and maintains a private Gitlab installation, along with a whole host of other core platform services for internal use. Gitlab is by far the most modular off-the-shelf product I've encountered outside of JFrog's Xray. Look at their official Helm chart: https://gitlab.com/gitlab-org/charts/gitlab https://gitlab.com/gitlab-org/charts/gitlab. Gitlab itself consists of 14 sub-charts and it also bundles 4 third-party sub-charts for object storage, a web proxy and ingress controller, certificate management, and the internal container registry. Gitlab without the third parties I believe consists of 15 distinct containers. I don't think it matches what most people think of when they hear "monolith." It is absolutely not a single process only communicating between components via function calls. Many of the Gitlab core services, such as Gitaly, are written in Go, as well, not Ruby, though they also have "gitaly-ruby" as a testing service that can be used by developers not comfortable with Go.
- native_samples 4y ago
  He mentions gitaly in the blog post.
- john_cogs 4y ago
  GitLab team member here. Sid addressed GitLab's use of Go in a comment on a previous discussion of this article from last week: https://news.ycombinator.com/item?id=31687289 https://news.ycombinator.com/item?id=31687289 He wrote: "We're moving the 20% of the app consuming 80% of the compute to Go." His comment also highlights another benefit of the way our core services are structured: reusability.
marcosdumay 4y ago
It's worse, because now you can't fix your bad boundaries, because there are different teams working on each side, and different projects under different PMs.

andrew_ 4y ago

Agreed. The kicker is knowing what those boundaries are and avoiding them. I really enjoy working with microservices for async tasks, and I think they're really well-suited to them. I really dislike inter-service dependencies. And of course, there's nothing wrong with a few, small monolith services. It's all about the right tool for the right job for me, and being too prescriptive to a particular architecture comes at a high cost.

ravenstine 4y ago

A better first approach would be to try and remove roadblock and bureaucracy that are encouraging developers to actually consider microservices. Those are of course not the only the only arguments for microservices, but I do think they're a big one that more often goes unsaid than the modularity. You can have modularity in a monolithic app, but for some reason many teams don't actually take modularity seriously; they ask that everything be object-oriented and call it a day, as if writing a `module` in Ruby means you've made things modular. Microservices become appealing when the process to have monolithic code reviewed and deployed is long and painful, since they are by definition small, decoupled, and simple to deploy. Of course it often doesn't quite work out that way, in which case a monolith would have worked anyway. But it's unlikely things will change in coding culture so long as there's the perverse incentive for businesses to encourage their engineers to use hacks out of expediency. Hacks inevitably make things really hard down the road, create mysterious problems that are hard to solve, and the solution is often to add bureaucracy or switch to microservice architecture.

ar_lan 4y ago
Yeah, I agree. I've worked with a variety of monoliths. The biggest complaint I've had from companies who utilize that architecture is that I shouldn't have to run entire pre and post pipelines to make a minor change to an internal function that only affects one package in the entire repository. This is solvable, but it is a problem - one that causes a lot of engineers to consider starting fresh in a new repo anyway. Multiple repositories inherently solve this for you - but bring about other problems. In my opinion - most projects don't need microservice architecture in the sense of separated binaries. Typically, they can be written with clear boundaries within the monolith such that, should the time come when scaling concerns are real for certain applications, then they can be ripped out as needed into their own binaries.
- potbelly83 4y ago
  Interesting question arises here. Given most shops are now using interpreted languages, is it possible to make a change to say a .py file and ONLY deploy that .py file to production? i.e. do incremental releases
  - ar_lan 4y ago
    I don't know if "most shops are now using interpreted languages" is an accurate statement :) (I actually don't know). But I think that's definitely an interesting idea (likely with some small levels of extra caution with some specific high profile files, like an API definition/endpoint or something).
    - potbelly83 4y ago
      Ha no worries, to be fair most of my working life has been spent at C++ shops. I guess Windows does something similar via dlls.
  - pojzon 4y ago
    The issue was rather about the fact that changing that single .py file can have impact on a service A and service B which depend on service C where the file was changed. In microservice world, you have to run e2e tests because thats the only place where you can really guarantee the system will work as you expect. Pacts wont solve that, integration test - the same, units w/e. Running a monolith test is overall faster than spinning 20 microservices, beinging every db to specific atate and running tests.

blobbers 4y ago

While that's true that the network calls should be function calls, it means you can scale horizontally in a lot easier fashion. Sure you've got waste at the small scale, but if you're building for webscale then you have to realize these are the trade offs one must make.

willcipriano 4y ago
I've seen several times where the overhead of the network request is dramatically higher than the work being preformed. The worst example was a validation microservice that did things like "is True" or "is greater than 5" one element at a time on a large object. So you'd have an object with 50 elements and it would make 50 requests to the service. This was in the context of a batch processing job that handled millions of items, so billions of requests end up being created. I tried to explain that each network request probably does a few thousand things like "if request_type == 'get'" during it's lifecycle on both sides of the transaction, but nobody got it and I quit soon after.
- brianwawok 4y ago
  A lot of devs just don’t care. They want to do a cool thing for their resume. This usually means there is no tech leadership that “gets it”.
  - runevault 4y ago
    Resume driven development is one of the most depressing things I keep running into in my career. Wanting to learn and grow is great, but forcing customers to suffer the repercussions of your learning because technology/design pattern x is in vogue is closing in on malpractice.
    - brianwawok 4y ago
      Until there is consequences, why not? Next time you interview with someone, ask why they converet a system to Microservices at their last job? If they don't have an actual reason, bin them.

ryanmcbride 4y ago

Yup I've seen it at just about every company I've worked for. A cluster of microservices that is really just a distributed monolith. Is it really a microservice if changing one part of it requires making echoing changes to every service downstream? Is it really a microservice if one part breaking causes the entire system to fail spectacularly? I feel like the pendulum is starting to swing the other way as companies learn that microservices aren't the end-all solution to their process problems, and that prioritizing new features every sprint and constantly pushing housekeeping, bugsquashing, and system improvements will result in a broken collection of microservices just as quickly as it resulted in a broken monolith.

411111111111111 4y ago
> Is it really a microservice if one part breaking causes the entire system to fail spectacularly? No, that's a distributed monolith not a microservice
cogman10 4y ago
> prioritizing new features every sprint and constantly pushing housekeeping, bugsquashing, and system improvements will result in a broken collection of microservices just as quickly as it resulted in a broken monolith. Now, HTF do you convince a C-Level of this truth? Seems like there is never any time for any sort of housekeeping until the house is literally on fire and the whole world is looking at our smoke pillar. Even then, they'll usually be like "Ok, take 2 weeks to fix all our architecture problems and then get back to work on new features!". As SOON as the fire is doused, they want to move on rather than cleaning and removing the flammable material and the flame that keep starting the fire.
- mixedCase 4y ago
  You don't, it's not his job to care, it's yours. And just like you don't ask permission for writing down a line of feature code, you don't ask permission for maintenance work. You just do it and schedule it accordingly to business needs to the best of your ability. And make sure not to apologize for doing your work or treat maintenance work as "unimportant" because that is how it will be perceived. It does get tricky when you have a manager defining tasks and you have yet to determine whether their job is to be a technical manager (and is adequately competent) or a nerd babysitter/translator. In the former case it's good to run maintenance projects by them if they need active, dedicated time as opposed to something you can mostly do during downtime. In case it's the other kind of "manager", and you've already tried and failed to convince them to prioritize basic stuff, you have to just put it inside other estimates and rope in the rest of your team to do the same and/or polish your résumé.

nine_k 4y ago

Why, SOA is a reasonable concept, and does help scaling. Say, billing, ETL and batch processing, and Web backends can live as separate services all right. Going with micro-services is another story, and it makes sense in a more narrow gamut of circumstances. Say, microservices may be a great fit for AWS Lambda-style deployments, but this assumes spiky, sparse load patterns.

thibauts 4y ago
You can deploy your whole monolith on one single lambda and it will scale the same. It may even help to keep it hot. Services, micro or not are only really useful when load factors are vastly different and you’re not serverless. Or when you have sufficiently many large teams and a sufficiently large codebase that a single deploy becomes unmanageable because too much communication / coordination is needed.

jasonwatkinspdx 4y ago

When the microservice craze took off a lot of people asked me for opinions on it, and said the same thing then I say now: it more or less boils down to Conway's law. If you have multiple teams that need to iterate and deploy independently the overhead may be worth it. But if you're just a small startup with a single unitary dev team, and that generally just deploys a new version of any changed services all as one lump, it's an insane amount of complexity and overhead.

revskill 4y ago

As long as microservices don't share database, then all is fine.

Srikanth 4y ago

This probably is the reason why it works in some settings. Anecdotally, as someone with more domain/functional knowledge and operations knowledge than knowledge of frameworks, I found microservices architecture with good functional test coverage a better way to deal with a team of programmers like me (basically, a typical team in an offshore IT consultancy building enterprise applications using SpringBoot and Node.JS). Ship the service as early as possible with available talent and then get someone really good with that programming language or framework to deal with performance bottlenecks within the microservice. I see it basically as a way to limit the blast radius of the applications. Of course, as you said, get the boundaries wrong and you have a bigger problem.

nonameiguess 4y ago

It kind of feels like Sid is lying through his teeth here, as a person who deploys and maintains a private Gitlab installation, along with a whole host of other core platform services for internal use. Gitlab is by far the most modular off-the-shelf product I've encountered outside of JFrog's Xray. Look at their official Helm chart: https://gitlab.com/gitlab-org/charts/gitlab https://gitlab.com/gitlab-org/charts/gitlab. Gitlab itself consists of 14 sub-charts and it also bundles 4 third-party sub-charts for object storage, a web proxy and ingress controller, certificate management, and the internal container registry. Gitlab without the third parties I believe consists of 15 distinct containers. I don't think it matches what most people think of when they hear "monolith." It is absolutely not a single process only communicating between components via function calls. Many of the Gitlab core services, such as Gitaly, are written in Go, as well, not Ruby, though they also have "gitaly-ruby" as a testing service that can be used by developers not comfortable with Go.

native_samples 4y ago
He mentions gitaly in the blog post.
john_cogs 4y ago
GitLab team member here. Sid addressed GitLab's use of Go in a comment on a previous discussion of this article from last week: https://news.ycombinator.com/item?id=31687289 https://news.ycombinator.com/item?id=31687289 He wrote: "We're moving the 20% of the app consuming 80% of the compute to Go." His comment also highlights another benefit of the way our core services are structured: reusability.

marcosdumay 4y ago

It's worse, because now you can't fix your bad boundaries, because there are different teams working on each side, and different projects under different PMs.