9 ms·

Hmm, i don't think our needs fall into any of those three. But our reasons may just be, as you say, the wrong reasons. We split up our services when the major

Hmm, i don't think our needs fall into any of those three. But our reasons may just be, as you say, the wrong reasons. We split up our services when the majority of changes made to a service can be made independently of everyone else. So then each of our services is a different codebase, and they each have a completely different rate of change from each other. You very rarely would ever make changes across all services as once, only when changing what's being communicated between some set of services. Some services are large, some are small, and many of them fall into the category of -- developed for a while then stabilized and now basically rarely every touched just works. So then the advantage is that you always have a small mental working set. You're focusing on a service at a time, have less to worry about breaking everything else with your changes. And then when you deploy, even if everything goes horribly wrong, it's just your one service that is down, and everything else is up and running, and you'll just have to process your queued messages once you came back online. And then of course each service is smaller, so less code, less tests, faster to compile, faster to run through the pipeline, faster to release. And you're only ever doing rolling deploys on a small sub-section of your infrastructure, never the whole thing at once.

AlexCoventry 5y ago
Modular interfaces are great, but why require communication over those interfaces to go via network traffic? Can't you just use a library/package?
- spaetzleesser 5y ago
  That's what I am always. People can't manage libraries so they think services will solve this problem. It seems to me that a lot of microservices are creating technical debt that will come due when the requirements change in a way that requires a big overhaul of the system.
- jmchuster 5y ago
  Because there is state?
  - anyfoo 5y ago
    I think the question was meant to be along the lines of: Why have that unit as a separate service then, instead of just locally as, e.g., a shared library, running in the same process? Unless you have the specific need of that piece of code not being in the same process/container/computer/network. Unless your linking time is prohibitively high (but even very large software takes minutes, not hours), the benefit to compile time should be the same. But maybe "redeploying" is more disruptive (I don't know, I haven't done web development for something approaching 10 years now), and depending on the language, problems in one part can crash the whole process. (But depending on the actual process model used, shouldn't it just restart? Unless the language is not sufficiently memory safe, e.g. it's C, and you get way worse problems with memory smashers.)
    - gravypod 5y ago
      Think of a situation where you have, for example, a database like Redis. If you need to run two systems that need to access that data there will need to be some process that maintains that data and allows other people to access it. You would have a hard time to make a resource efficient distributed in-process key-value store. You could have an in-process library but this would cause your two programs to have either: 1. Store different data 2. All writes would need to be distributed to all processes (looping back to needing a network connection).
      - anyfoo 5y ago
        Yeah, that makes sense for the data store, but don't microservices split up much more? Of course this could quickly devolve into a conversation about "at what point is it microservices".
        gravypod 5y ago
        > Of course this could quickly devolve into a conversation about "at what point is it microservices". Yep, exactly. I've built more than one microservice in my time that loads up a few GB of data into RAM and efficiently queries it for a lot of services. At that point the line between DB and service gets blurry. But, essentially, the TCP connection between your services isn't necessarily a big issue. If there is some logic/operation/access that is complex it makes sense to chuck a language-independent interface in front of it for most use cases. There are some where performance trumps maintenance but where it doesn't I think this abstraction is helpful.
- gravypod 5y ago
  Some reasons may be: 1. Multiple languages/tech stacks. Have something like protobufs and now any language can "import" your library instead of it just being things that can handle cffi. 2. State management. For example, redis couldn't be an in-process library. There are in-process kv stores but that's not the idea of redis. 3. Transparent monitoring for all languages. I can passively detect all retries, latencies, request counts, etc since there is a protocol. 4. Centralized fixes. Similarly to dynamic linking I can now update a single component and "fix the world" rather than redeploying every binary. 5. Canaries: I can deploy version N+1 to a mirror of production traffic and make sure it doesn't explode. Then I can route 1%, 10%, 50%, 100% of traffic and check those metrics as well. 6. If something goes horribly wrong I can rewrite the entire service for performance or maintenance or whatever within a tightly bounded amount of time. Discord has many blog posts about doing this for some of their services. Great talk: https://www.youtube.com/watch?v=-UKEPd2ipEk https://www.youtube.com/watch?v=-UKEPd2ipEk
  - erik_seaberg 5y ago
    > Multiple languages/tech stacks. This is what I'm excited about--enabling a team to use the most powerful tools that address their own problems, rather than whichever Blub is approachable enough for the entire staff.
    - throwawaaarrgh 5y ago
      But this is a problem at large scale. Each language and stack carries a variety of overhead. The more of them you have, the higher the overhead is. Say you run a factory, and everyone needs a hammer. When there's only two teams, two kinds of hammer is fine. But fast forward to 10 teams. Can you always get all of those hammers when you need them? What if team #3 needs extra support? You need to find a new engineer who knows hammer #3, or take an engineer off of team #6 and train them on how to use hammer #3. And if all the hammers need a slight modification specific to that company, you need to make 10 different modifications. And how will all the teams benefit from shared hammer tips-n-tricks if they all use different hammers? There's a lot more overhead than you think at first.
rualca 5y ago
>>So then the advantage is that you always have a small mental working set. You don't need microservices architectures for that. Even a clean architecture on a monolith ensures that you can compartmentalize the problem domain. At most, just refactor out that responsibility into a library/component/module.
- jmchuster 5y ago
  It's a little bit hard to cleanly separate each grouping of database+cache+background tasks+apis+web ui.

AlexCoventry 5y ago

Modular interfaces are great, but why require communication over those interfaces to go via network traffic? Can't you just use a library/package?

spaetzleesser 5y ago
That's what I am always. People can't manage libraries so they think services will solve this problem. It seems to me that a lot of microservices are creating technical debt that will come due when the requirements change in a way that requires a big overhaul of the system.
jmchuster 5y ago
Because there is state?
- anyfoo 5y ago
  I think the question was meant to be along the lines of: Why have that unit as a separate service then, instead of just locally as, e.g., a shared library, running in the same process? Unless you have the specific need of that piece of code not being in the same process/container/computer/network. Unless your linking time is prohibitively high (but even very large software takes minutes, not hours), the benefit to compile time should be the same. But maybe "redeploying" is more disruptive (I don't know, I haven't done web development for something approaching 10 years now), and depending on the language, problems in one part can crash the whole process. (But depending on the actual process model used, shouldn't it just restart? Unless the language is not sufficiently memory safe, e.g. it's C, and you get way worse problems with memory smashers.)
  - gravypod 5y ago
    Think of a situation where you have, for example, a database like Redis. If you need to run two systems that need to access that data there will need to be some process that maintains that data and allows other people to access it. You would have a hard time to make a resource efficient distributed in-process key-value store. You could have an in-process library but this would cause your two programs to have either: 1. Store different data 2. All writes would need to be distributed to all processes (looping back to needing a network connection).
    - anyfoo 5y ago
      Yeah, that makes sense for the data store, but don't microservices split up much more? Of course this could quickly devolve into a conversation about "at what point is it microservices".
      - gravypod 5y ago
        > Of course this could quickly devolve into a conversation about "at what point is it microservices". Yep, exactly. I've built more than one microservice in my time that loads up a few GB of data into RAM and efficiently queries it for a lot of services. At that point the line between DB and service gets blurry. But, essentially, the TCP connection between your services isn't necessarily a big issue. If there is some logic/operation/access that is complex it makes sense to chuck a language-independent interface in front of it for most use cases. There are some where performance trumps maintenance but where it doesn't I think this abstraction is helpful.
gravypod 5y ago
Some reasons may be: 1. Multiple languages/tech stacks. Have something like protobufs and now any language can "import" your library instead of it just being things that can handle cffi. 2. State management. For example, redis couldn't be an in-process library. There are in-process kv stores but that's not the idea of redis. 3. Transparent monitoring for all languages. I can passively detect all retries, latencies, request counts, etc since there is a protocol. 4. Centralized fixes. Similarly to dynamic linking I can now update a single component and "fix the world" rather than redeploying every binary. 5. Canaries: I can deploy version N+1 to a mirror of production traffic and make sure it doesn't explode. Then I can route 1%, 10%, 50%, 100% of traffic and check those metrics as well. 6. If something goes horribly wrong I can rewrite the entire service for performance or maintenance or whatever within a tightly bounded amount of time. Discord has many blog posts about doing this for some of their services. Great talk: https://www.youtube.com/watch?v=-UKEPd2ipEk https://www.youtube.com/watch?v=-UKEPd2ipEk
- erik_seaberg 5y ago
  > Multiple languages/tech stacks. This is what I'm excited about--enabling a team to use the most powerful tools that address their own problems, rather than whichever Blub is approachable enough for the entire staff.
  - throwawaaarrgh 5y ago
    But this is a problem at large scale. Each language and stack carries a variety of overhead. The more of them you have, the higher the overhead is. Say you run a factory, and everyone needs a hammer. When there's only two teams, two kinds of hammer is fine. But fast forward to 10 teams. Can you always get all of those hammers when you need them? What if team #3 needs extra support? You need to find a new engineer who knows hammer #3, or take an engineer off of team #6 and train them on how to use hammer #3. And if all the hammers need a slight modification specific to that company, you need to make 10 different modifications. And how will all the teams benefit from shared hammer tips-n-tricks if they all use different hammers? There's a lot more overhead than you think at first.

rualca 5y ago

>>So then the advantage is that you always have a small mental working set. You don't need microservices architectures for that. Even a clean architecture on a monolith ensures that you can compartmentalize the problem domain. At most, just refactor out that responsibility into a library/component/module.

jmchuster 5y ago
It's a little bit hard to cleanly separate each grouping of database+cache+background tasks+apis+web ui.