6 ms·

Hahaha. I have been saying the same for years but have either been punished by my managers or mercilessly downvoted on HN. Somehow mentioning that microservice

Hahaha. I have been saying the same for years but have either been punished by my managers or mercilessly downvoted on HN. Somehow mentioning that microservices might not be perfect solution in every case triggers a lot of people. I have actually helped save at least one project in a huge bank which got rolled from 140 services into one. The team got also scaled down to third of its size but was able to work on this monolithic application way more efficiently. Also reliability, which was huge issue before the change, improved dramatically. When I joined, I have observed 6 consecutive failed deployments. Each took entire week to prepare and entire weekend to execute (with something like 40 people on bridge call). When I left I have observed 50 consecutive successful deployments, each requiring 1h to prepare (basically meeting to discuss and approve the change) and 2h of a single engineer to prepare and execute using automation. Most projects absolutely don't need microservices. Breaking anything apart brings inefficiencies of having to manage multiple things. Your people now spend time managing applications rather than writing actual business logic. You have to have really mature process to bring those inefficiencies down. If you want to "do microservices" you have to precisely know what kind of benefits you are after. Because the benefits better be higher than the costs or you are just sabotaging your project. There are actually ways to manage huge monolithic application that don't require each team to have their own repository, ci/cd, binary, etc. How do you think things like Excel or PhotoShop have been developed? It is certainly too large for a single team to handle.

Mandatum 6y ago
I think my biggest gripe with orgs that adopt microservices is they don't build out any of the testing, CI/CD, monitoring and debugging workflows to support it. It goes from shitty, slow monolithic application that super-pro can debug in a few minutes to.. Slow, shitty disparate services that are managed by different teams who don't share anything, suddenly you've got a Cuckoo's Egg situation where 1 guy needs to get access to all the things to find out what the fuck is happening. Or you just accept it's shitty and slow, and pay a consultancy to rebuild it in New Thing 2.0 in 8 years when accounting forget about the last 3 rebuilds.
- lmilcin 6y ago
  That is EXACTLY what I have observed, time and time again. If you have trouble managing ONE application what makes you think you will be better at managing multiple? Also, running distributed system is way more complicated than having all logic in a single application. Ideally you want to delay switching to distributed system until it is inevitable that you are not going to be able to fulfill the demand using monolithic service. If your application has problems, don't move to microservices. Remove all unnecessary tasks and let your team focus on solving the problems first and then automate all your development, testing, deployment and maintenance processes. Or call me and I can help you diagnose and plan:)
  - bulleyashah 6y ago
    Monoliths are also distributed systems and will run on multiple hosts, most probably co-ordinating on some sort of state (and that state management will need to take care of concurrency, consistency). Some hosts will go down. Service traffic will increase. I understand your point. You are using distributed in the sense of "how is one big work distributed", you probably also hate overly "Object Oriented code" for similar reasons. But distributed systems is a well understood thing in the industry. If I call you and you tell me this, then you're directly responsible for hurting how successful I would be by giving me a misleading sense of what a distributed systems is.
    - pjc50 6y ago
      > Monoliths are also distributed systems and will run on multiple hosts ... not necessarily. Although the big SPOF monolith has gone out of fashion, do not underestimate the throughput possible from one single very fast server.
      - bulleyashah 6y ago
        Well, no matter how fast a single server is, it can't keep getting faster. You might shoot yourself in the foot by optimizing only for single servers because eventually you'll need horizontal scaling and it's better to think about it in the beginning of your architecture.
measure2xcut1x 6y ago
> There are actually ways to manage huge monolithic application that don't require each team to have their own repository, ci/cd, binary, etc. Would be interested to hear about some of these.
- fragile_frogs 6y ago
  A monorepo I guess. It looks like Microsoft is using a monorepo for their office applications (https://rushjs.io/ https://rushjs.io/) and you could do the same thing for node.js using yarn workspaces/lerna/rush. Each "Microservice" could live in a separate package which you can import and bundle into single executable. Elixir has "Umbrella Projects": https://elixirschool.com/en/lessons/advanced/umbrella-projects/ https://elixirschool.com/en/lessons/advanced/umbrella-projec... Rust/Cargo has workspaces: https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
- RegW 6y ago
  Well for starters, each component's API can be published and versioned separately from its implementation. The build of a component would only have access to the API's of the other components (and this can include not having knowledge of the container it runs in). The implementation can then change rapidly, with the API that the other teams develop against moving more slowly. Even so, code reviews can be critical. The things to look out for (and block if possible) are hidden or poorly defined parameters like database connections/transactions, thread local storage and general bag parameters. In some languages dependency injection should be useful here. Unfortunately DI tools like Spring can actually expose the internals of components, introduce container based hidden parameters and usually end up being a versioned dependency of every component.
- lmilcin 6y ago
  The basic technique is to use modular architecture. You divide your application into separate problems each represented by one or more modules. You create API for these modules to talk to each other. You also create some project-wide guidelines for application architecture, so that the modules coexist as good neighbors. You then have separate teams responsible for one or more modules. If your application is large enough you might consider building some additional internal framework, for example plugin mechanism. For example, if your application is an imaginary banking system that takes care of users' accounts, transactions and products they have, you might have some base framework (which is flows of data in the application, events like pre/post date change, etc.) and then you might have different products developed to subscribe to those flows of data or events and act upon the rest of the system through internal APIs.
MattyMc 6y ago
I don't think you can downvote on HackerNews. That's reddit.
- kevsim 6y ago
  You can, you just need 501 karma first. Seems like someone wanted to prove you wrong since you've been downvoted.
  - MattyMc 6y ago
    Haha seems a few people wanted to prove me wrong :) Didn't know that!
- theshrike79 6y ago
  You need 500 karma for the downvote button to appear: https://jacquesmattheij.com/the-unofficial-hn-faq/#karma https://jacquesmattheij.com/the-unofficial-hn-faq/#karma
sublimefire 6y ago
LOL I'm in talks of migrating microservices into a monolith purely because there is so much overhead at managing it. You need at least a couple of people to keep it in place. And the latter means the company is prepared to kill the product even when there are actual customers plus new ones are coming. Microservices make sense when you have millions of users and there is a need to quickly scale horizontally. Or when you have a zillion of developers which probably means that your product is huge. Or when you are building a global service from the get go and get funded by a VC.
delecti 6y ago
A sister team at work (same manager, separate sprints) split a reasonable component into three microservices. One with business logic, one which talks to an external service, and one which presents an interface to outside clients. They then came to own the external service as well. So now they have 4 micro services when one or two would have been plenty. I don't hesitate to poke fun when that inevitably brings them frustration. Another team we work with has their infrastructure split across a half-dozen or more microservices, and then they use those names in design discussions as though we should have any idea which sub component they're talking about. Microservices are a tool that fits some situations, but they can be used so irresponsibly.
ex_amazon_sde 6y ago
> I have been saying the same for years but have either been punished by my managers or mercilessly downvoted on HN. Ex Amazon SDE here. I've been saying many times that Amazon tends to have the right granularity of services: roughly 1 or 2 services for a team. A team that maintains the service in production autonomously and deploys updates without having to sync up with other teams. [Disclaimer: I'm not talking about AWS]
obviouslynotme 6y ago
I can understand Résumé-Driven Development. Our industry is famous for its "Must have (X+2) years of (X year old technology)" job requirements.

Mandatum 6y ago

I think my biggest gripe with orgs that adopt microservices is they don't build out any of the testing, CI/CD, monitoring and debugging workflows to support it. It goes from shitty, slow monolithic application that super-pro can debug in a few minutes to.. Slow, shitty disparate services that are managed by different teams who don't share anything, suddenly you've got a Cuckoo's Egg situation where 1 guy needs to get access to all the things to find out what the fuck is happening. Or you just accept it's shitty and slow, and pay a consultancy to rebuild it in New Thing 2.0 in 8 years when accounting forget about the last 3 rebuilds.

lmilcin 6y ago
That is EXACTLY what I have observed, time and time again. If you have trouble managing ONE application what makes you think you will be better at managing multiple? Also, running distributed system is way more complicated than having all logic in a single application. Ideally you want to delay switching to distributed system until it is inevitable that you are not going to be able to fulfill the demand using monolithic service. If your application has problems, don't move to microservices. Remove all unnecessary tasks and let your team focus on solving the problems first and then automate all your development, testing, deployment and maintenance processes. Or call me and I can help you diagnose and plan:)
- bulleyashah 6y ago
  Monoliths are also distributed systems and will run on multiple hosts, most probably co-ordinating on some sort of state (and that state management will need to take care of concurrency, consistency). Some hosts will go down. Service traffic will increase. I understand your point. You are using distributed in the sense of "how is one big work distributed", you probably also hate overly "Object Oriented code" for similar reasons. But distributed systems is a well understood thing in the industry. If I call you and you tell me this, then you're directly responsible for hurting how successful I would be by giving me a misleading sense of what a distributed systems is.
  - pjc50 6y ago
    > Monoliths are also distributed systems and will run on multiple hosts ... not necessarily. Although the big SPOF monolith has gone out of fashion, do not underestimate the throughput possible from one single very fast server.
    - bulleyashah 6y ago
      Well, no matter how fast a single server is, it can't keep getting faster. You might shoot yourself in the foot by optimizing only for single servers because eventually you'll need horizontal scaling and it's better to think about it in the beginning of your architecture.

measure2xcut1x 6y ago

> There are actually ways to manage huge monolithic application that don't require each team to have their own repository, ci/cd, binary, etc. Would be interested to hear about some of these.

fragile_frogs 6y ago
A monorepo I guess. It looks like Microsoft is using a monorepo for their office applications (https://rushjs.io/ https://rushjs.io/) and you could do the same thing for node.js using yarn workspaces/lerna/rush. Each "Microservice" could live in a separate package which you can import and bundle into single executable. Elixir has "Umbrella Projects": https://elixirschool.com/en/lessons/advanced/umbrella-projects/ https://elixirschool.com/en/lessons/advanced/umbrella-projec... Rust/Cargo has workspaces: https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html
RegW 6y ago
Well for starters, each component's API can be published and versioned separately from its implementation. The build of a component would only have access to the API's of the other components (and this can include not having knowledge of the container it runs in). The implementation can then change rapidly, with the API that the other teams develop against moving more slowly. Even so, code reviews can be critical. The things to look out for (and block if possible) are hidden or poorly defined parameters like database connections/transactions, thread local storage and general bag parameters. In some languages dependency injection should be useful here. Unfortunately DI tools like Spring can actually expose the internals of components, introduce container based hidden parameters and usually end up being a versioned dependency of every component.
lmilcin 6y ago
The basic technique is to use modular architecture. You divide your application into separate problems each represented by one or more modules. You create API for these modules to talk to each other. You also create some project-wide guidelines for application architecture, so that the modules coexist as good neighbors. You then have separate teams responsible for one or more modules. If your application is large enough you might consider building some additional internal framework, for example plugin mechanism. For example, if your application is an imaginary banking system that takes care of users' accounts, transactions and products they have, you might have some base framework (which is flows of data in the application, events like pre/post date change, etc.) and then you might have different products developed to subscribe to those flows of data or events and act upon the rest of the system through internal APIs.

MattyMc 6y ago

I don't think you can downvote on HackerNews. That's reddit.

kevsim 6y ago
You can, you just need 501 karma first. Seems like someone wanted to prove you wrong since you've been downvoted.
- MattyMc 6y ago
  Haha seems a few people wanted to prove me wrong :) Didn't know that!
theshrike79 6y ago
You need 500 karma for the downvote button to appear: https://jacquesmattheij.com/the-unofficial-hn-faq/#karma https://jacquesmattheij.com/the-unofficial-hn-faq/#karma

sublimefire 6y ago

LOL I'm in talks of migrating microservices into a monolith purely because there is so much overhead at managing it. You need at least a couple of people to keep it in place. And the latter means the company is prepared to kill the product even when there are actual customers plus new ones are coming. Microservices make sense when you have millions of users and there is a need to quickly scale horizontally. Or when you have a zillion of developers which probably means that your product is huge. Or when you are building a global service from the get go and get funded by a VC.

delecti 6y ago

A sister team at work (same manager, separate sprints) split a reasonable component into three microservices. One with business logic, one which talks to an external service, and one which presents an interface to outside clients. They then came to own the external service as well. So now they have 4 micro services when one or two would have been plenty. I don't hesitate to poke fun when that inevitably brings them frustration. Another team we work with has their infrastructure split across a half-dozen or more microservices, and then they use those names in design discussions as though we should have any idea which sub component they're talking about. Microservices are a tool that fits some situations, but they can be used so irresponsibly.

ex_amazon_sde 6y ago

> I have been saying the same for years but have either been punished by my managers or mercilessly downvoted on HN. Ex Amazon SDE here. I've been saying many times that Amazon tends to have the right granularity of services: roughly 1 or 2 services for a team. A team that maintains the service in production autonomously and deploys updates without having to sync up with other teams. [Disclaimer: I'm not talking about AWS]

obviouslynotme 6y ago

I can understand Résumé-Driven Development. Our industry is famous for its "Must have (X+2) years of (X year old technology)" job requirements.