3 ms·

I have spent more time than I'm comfortable with over the last couple of years railing against microservices. But this seems like a classic case where microserv

I have spent more time than I'm comfortable with over the last couple of years railing against microservices. But this seems like a classic case where microservices are needed. Each team can handle their own writing/validation and expose them to themselves and other services(including the global service). Of course you run into issues with transaction occurring across multiple databases but these problems are hard but solvable. > All incoming writes need to go into a centralized log, such as Kafka, and then from there the various databases can pull what they need, with each team making its own decisions about what it needs from that central log. This sounds crazy. I don't know any large companies that have successfully implemented it. This is basically arguing for a giant central database across the entire company. Good luck getting the 300 people necessary into a room and agreeing on a schema. P.S. I don't know if he's using database as a shorthand for a service. If he is then you can ignore everything I've written.

inimino 7y ago
I'm not sure I'd call it microservices, since they could be monolithic inside each organization, but I had the same thought. Just spend the time to agree on a standardized API that everyone can support, and then the central API just proxies to the appropriate subsidiary. I wonder why they haven't done it that way.
- ThePadawan 7y ago
  Personal experience from a past job: Coming from a clunky, overused but underdesigned monolithic structure, the work to standardize inter-team/inter-department communication is really hard. In general, the easiest thing is to overspecify those APIs and fight anyone who wants to simplify them. Complicated work looks good on a CV, after all. So now you have hard-to-implement, full-of-cruft API designs. At that point, teams realize that the easiest thing is to work around them. And off you go into splintered components which ignore the standard APIs as much as possible. Turns out that's much easier, and you deliver results with higher velocity! From far enough away, you can just see "more standard APIs => higher velocity", so obviously you keep doing that, right?
  - huffmsa 7y ago
    Developers and product people have a hard time apply the idea of only building the simplest, most minimal "thing" the consumer needs first and gradually iterating it when the developers and other internal personnel are the consumer. An internal API is just as much a revenue driving product as an external one.
- JamesBarney 7y ago
  I'm honestly not sure where mini-services, micro-services, nano-services, and service oriented architecture begin or end relative to each other. So a micro service is 2 pizzas. Maybe for just a plain service you can have up to a a pizza party's worth of people. But you can only have up to 10 nano services per sandwich before they become pico-services?
- pjc50 7y ago
  What might be easiest is "federated" or "decentralised" - since coordination is prohibitively difficult here, build a set of pieces independently. The truth is that they behave as separate companies in a Coase theory-of-the-firm sense. So treat them that way from an IT perspective.
pytester 7y ago
This is why I always say that microservices are a technical fix for a political problem. Technically they're harder to manage and often come at a high technical cost that you should never attempt to pay if you don't have to, but it's an effective way of managing the often extreme coordination costs that exist in large, dysfunctional companies that simply don't exist elsewhere. I used to rail against microservices for years (working in startups) until I finally worked at a large dysfunctional corporation trying to run a lot of complex processes that spanned a lot of tightly coupled departments and then I started to see how they made sense. It also gave me an appreciation of the problems Bezos recognized when he tried to decouple the organizational fiefdoms under his control.
- colechristensen 7y ago
  organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations At startups you don't need Conway's law, not enough people to be relevant. At large companies software and people organizations tend to match or in the case you and parent are talking about need to match. I'm willing to believe that things don't necessarily have to work this way, but I haven't seen it. https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law
- JamesBarney 7y ago
  100% agree but I use the term organizational instead of political because I feel like "political" hints that it's only necessary because of the organization's dysfunction and I believe microservices are useful even when the organization is functioning well.(though maybe the point where it makes more sense to transition is later in a well functioning organization)
  - retiredcoder 7y ago
    It is somewhat limiting the fact that in our industry, challenges related to social and human factors are either dismissed, denied, or taken under negative tone. Social sciences and political philosophy could really benefit how we treat ourselves daily, in my opinion. However, it seems that if it is not “technical” is dysfunctional. PS: I am not saying that’s my take from your post, just something I observe in general. Heck, even I use the term political to signal dysfunctional work. :/
amelius 7y ago
> Of course you run into issues with transaction occurring across multiple databases but these problems are hard but solvable. The only thing you need to do to fix this is run all the services on the same DB. > This sounds crazy. I don't know any large companies that have successfully implemented it. This is basically arguing for a giant central database across the entire company. Good luck getting the 300 people necessary into a room and agreeing on a schema. You don't need every service to use the same schema. You only need transactions that span all services. They can use any data schema they want. A single DB is only used for the ACID guarantees.
treis 7y ago
>But this seems like a classic case where microservices are needed. I dont think that will help. The fundamental problem is the amount of work necessary to begin producing results. Take the POS talking directly to the DB. What you want to build is a unified API but to do that you first need to write a new POS system and replace a bunch of physical devices. The problem is how do you convince the business folks to spend years and millions on something that will save/produce the company 0 dollars? The path to do something like what's in the article is to first spend a year or so digging into code and finding every single issue like the POS one. Then you spend years fixing all of those issues. Then you spend more years actually building what you want. As a consulting firm if you come in and bid $100 million and 7 years while everyone else does $30 million over 3, well, you lose. So thats what you bid. Who cares if it goes tits up? You collect your fat bonuses for 18 months and then move onto your next firm/position. The inevitable failure can be blamed on the person who takes over.

inimino 7y ago

I'm not sure I'd call it microservices, since they could be monolithic inside each organization, but I had the same thought. Just spend the time to agree on a standardized API that everyone can support, and then the central API just proxies to the appropriate subsidiary. I wonder why they haven't done it that way.

ThePadawan 7y ago
Personal experience from a past job: Coming from a clunky, overused but underdesigned monolithic structure, the work to standardize inter-team/inter-department communication is really hard. In general, the easiest thing is to overspecify those APIs and fight anyone who wants to simplify them. Complicated work looks good on a CV, after all. So now you have hard-to-implement, full-of-cruft API designs. At that point, teams realize that the easiest thing is to work around them. And off you go into splintered components which ignore the standard APIs as much as possible. Turns out that's much easier, and you deliver results with higher velocity! From far enough away, you can just see "more standard APIs => higher velocity", so obviously you keep doing that, right?
- huffmsa 7y ago
  Developers and product people have a hard time apply the idea of only building the simplest, most minimal "thing" the consumer needs first and gradually iterating it when the developers and other internal personnel are the consumer. An internal API is just as much a revenue driving product as an external one.
JamesBarney 7y ago
I'm honestly not sure where mini-services, micro-services, nano-services, and service oriented architecture begin or end relative to each other. So a micro service is 2 pizzas. Maybe for just a plain service you can have up to a a pizza party's worth of people. But you can only have up to 10 nano services per sandwich before they become pico-services?
pjc50 7y ago
What might be easiest is "federated" or "decentralised" - since coordination is prohibitively difficult here, build a set of pieces independently. The truth is that they behave as separate companies in a Coase theory-of-the-firm sense. So treat them that way from an IT perspective.

pytester 7y ago

This is why I always say that microservices are a technical fix for a political problem. Technically they're harder to manage and often come at a high technical cost that you should never attempt to pay if you don't have to, but it's an effective way of managing the often extreme coordination costs that exist in large, dysfunctional companies that simply don't exist elsewhere. I used to rail against microservices for years (working in startups) until I finally worked at a large dysfunctional corporation trying to run a lot of complex processes that spanned a lot of tightly coupled departments and then I started to see how they made sense. It also gave me an appreciation of the problems Bezos recognized when he tried to decouple the organizational fiefdoms under his control.

colechristensen 7y ago
organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations At startups you don't need Conway's law, not enough people to be relevant. At large companies software and people organizations tend to match or in the case you and parent are talking about need to match. I'm willing to believe that things don't necessarily have to work this way, but I haven't seen it. https://en.wikipedia.org/wiki/Conway%27s_law https://en.wikipedia.org/wiki/Conway%27s_law
JamesBarney 7y ago
100% agree but I use the term organizational instead of political because I feel like "political" hints that it's only necessary because of the organization's dysfunction and I believe microservices are useful even when the organization is functioning well.(though maybe the point where it makes more sense to transition is later in a well functioning organization)
- retiredcoder 7y ago
  It is somewhat limiting the fact that in our industry, challenges related to social and human factors are either dismissed, denied, or taken under negative tone. Social sciences and political philosophy could really benefit how we treat ourselves daily, in my opinion. However, it seems that if it is not “technical” is dysfunctional. PS: I am not saying that’s my take from your post, just something I observe in general. Heck, even I use the term political to signal dysfunctional work. :/

amelius 7y ago

> Of course you run into issues with transaction occurring across multiple databases but these problems are hard but solvable. The only thing you need to do to fix this is run all the services on the same DB. > This sounds crazy. I don't know any large companies that have successfully implemented it. This is basically arguing for a giant central database across the entire company. Good luck getting the 300 people necessary into a room and agreeing on a schema. You don't need every service to use the same schema. You only need transactions that span all services. They can use any data schema they want. A single DB is only used for the ACID guarantees.

treis 7y ago

>But this seems like a classic case where microservices are needed. I dont think that will help. The fundamental problem is the amount of work necessary to begin producing results. Take the POS talking directly to the DB. What you want to build is a unified API but to do that you first need to write a new POS system and replace a bunch of physical devices. The problem is how do you convince the business folks to spend years and millions on something that will save/produce the company 0 dollars? The path to do something like what's in the article is to first spend a year or so digging into code and finding every single issue like the POS one. Then you spend years fixing all of those issues. Then you spend more years actually building what you want. As a consulting firm if you come in and bid $100 million and 7 years while everyone else does $30 million over 3, well, you lose. So thats what you bid. Who cares if it goes tits up? You collect your fat bonuses for 18 months and then move onto your next firm/position. The inevitable failure can be blamed on the person who takes over.