11 ms·

Service mesh use cases (2020)

135 points by biggestlou 4y ago

asim 4y ago
Article is from 2020. Please add to title.
- dang 4y ago
  Added. Thanks!
samsquire 4y ago
Thanks for this. I have never deployed a service mesh or used one but I am designing something similar at the code layer. It is designed to route between server components. That is, at the architecture between threads in a multithreaded system. The problem I want to solve is that I want architecture to be trivially easy to change with minimal code changes. This is the promise and allure of enterprise service buses and messaging queues and probably Spring. I have managed RabbitMQ and I didn't enjoy it. If I want a system that can scale up and down and that multiples of any system object can be introduced or removed without drastic rewrites. I would like to decouple bottleneck from code and turn it into runtime configuration. My understanding of things such as Traefik and istio is that they are frustrating to set up. Specifically I am working on designing interthread communication patterns for multithreaded software. How do you design an architecture that is easy to change, scales and is flexible? I am thinking of a message routing definition format that is extremely flexible and allows any topology to be created. https://github.com/samsquire/ideas4#526-multiplexing-setting-format-and-routing-operation https://github.com/samsquire/ideas4#526-multiplexing-setting... I think there is application of the same pattern to the network layer too. Each communication event has associated with it an environment of keyvalues that look similar to this: petsserver1 container1 thread3 socket5 user563 ingestionthread1 These can be used to route to keyspace ranges (such as particular users to tenant shards or load balance) to other components. For example users1-1000 are handled by petsserver1 and socket5 is associated with thread3. In other words: changing the RabbitMQ routing settings doesn't change the architecture of your software. You need to change the architecture of the software to match the routing configuration. But what if you changed the routing configuration and the application architecture changed to match?
- CraigJPerry 4y ago
  >> But what if you changed the routing configuration and the application architecture changed to match? If there were 3 ways to categorise scaling (there’s more than this in reality) then they might be vertical, horizontal then distributed. You’re describing an architecture that’s in the horizontal scaling world view. You’re not in vertical because you’re using higher powered (but slower) strategies like active routing for comms between components where in vertical you’d have configurable queues but no routing layer. You’re not in distributed scaling mode because your routing is assuming consistent latency and consistent bandwidth behaviours. I don’t think one architecture to rule them all is a solvable problem. I’d heartily and very gratefully welcome being proven wrong on this.
  - samsquire 4y ago
    Thanks for your comment. It's definitely food for thought. You remind me of fallacies of distributed computing by mentioning consistent latency and bandwidth. https://en.m.wikipedia.org/wiki/Fallacies_of_distributed_computing https://en.m.wikipedia.org/wiki/Fallacies_of_distributed_com... I'm still at the design stage. Those architectures you describe, I am hoping there is a representation that can describe many architectures. There's probably architecture's I am yet to think of that are unrepresentable with my format. Going from 1 to N or removing, adding a layer should be automatable. That's my hope anyway. I want everything to wire itself automatically. I am trying to come up with a data structure that can represent architecture. I am trying to do what inversion of control containers do per request but for architecture. In Inversion of control containers you specify a scope of an object that is instantiated for a scope such as for a request or for a session. I want that for architecture.
    - CraigJPerry 4y ago
      it’s such a fundamental problem space and with such a rich diversity of possible solutions that at a minimum you’re going to create something seriously useful for a subset of types of application. But it’d be transformational for computing if you cracked the whole problem. I hope you do. I do like your idea of outsourcing the wiring (an error prone, detail heavy task) away from humans.
NovemberWhiskey 4y ago
Now imagine you have something that has the complexity and change volume of a distributed control plane bringing together load-balancing, service advertisement, public key infrastructure, and software defined networking, and then try to imagine running it at the same reliability as your DNS. Also: proxies, proxies everywhere, as far as the eye can see.
- zidad 4y ago
  And in addition to that, all of those immediately becoming the same centralized single point of failure. What could possibly go wrong (on high load)? ;p
  - jspdown 4y ago
    In most implementations this is not the case. Service Meshes tend to either follow a sidecar or a DaemonSet approach. You don't have a single proxy, people usually complain about the exact opposite.
- darkwater 4y ago
  This is the production ready part that people will usually discover later on their own skin...
Scubabear68 4y ago
I’ve only read about Service Mesh, my impression was that it seems to add an awful lot of processes and complexity just to make developer’s lives slightly easier. Maybe I’m wrong but it almost feels like busy work for DevOps. Is my first impression wrong? Is this the right way to architect systems in some use cases, and if so what are they?
- MoOmer 4y ago
  Many of the use cases described in the post are solved by service meshes. So, in my opinion, the questions are introspective: - “Do I have enough context to know what problem those solutions are solving, and to at least appreciate the problem space to understand why someone may solve it like this?” - “Do I have or perceive those problem to impact my infrastructure/applications?” - “Does the solution offered by the use cases described appeal to me?” If yes at the end, then one potential implementation is a service mesh. A lot of these are solved out-of-the-box with Hashicorp’s Nomad/Consul/Vault pairing, for example!
  - nazka 4y ago
    I like how you frame the questions. How many times people pick a technology without answering them? Even having some knowledge in them. I am wondering does Nomad/Consul continue to scale after some level?
    - chromatin 4y ago
      I don't know about Consul, but Nomad has been scaled to 2,000,000 containers on >6000 hosts https://www.hashicorp.com/c2m https://www.hashicorp.com/c2m
  - remram 4y ago
    It is true that a lot of those use cases are covered by "basic" Kubernetes (or Nomad) without the addition of Istio or similar, e.g. service discovery, load-balancing, circuit-breaking, autoscaling, blue-green, isolation, health checking... Adding a service mesh onto Kubernetes seems to bring a lot of complexity for a few benefits (80% of the effort for the last 20% sort of deal).
jpdb 4y ago
I just wrote something extremely similar, but it's only internal right now. I personally find that the service mesh value-prop is hard to justify for a serverless stack (mostly Cloud Run, but AWS Lambda too probably), and in situations where your services are mostly all in the same language and you can bake the features into libraries that are much easier to import. Observability is a great example of this. In serverless-land, you're already getting the standard HTTP metrics (ex request count, response codes, latency, etc), tracing, and standard HTTP request logging "for free."
- davewritescode 4y ago
  > I personally find that the service mesh value-prop is hard to justify for a serverless stack (mostly Cloud Run, but AWS Lambda too probably), and in situations where your services are mostly all in the same language and you can bake the features into libraries that are much easier to import. If you’re running server less you already have 90% of what you’d get from a service mesh. I will tell you that having seen what happens in big companies, baking distributed concerns into libraries always ends in disaster long after you’re gone. When you have a piece of code deployed in 200 separate apps, every change requires tons of project management.
4pkjai 4y ago
I really did not enjoy dealing with our service mesh at the last place I worked.
- jspdown 4y ago
  Out of curiosity, was it something built internally? Or were you relying on a public solution?
  - 4pkjai 4y ago
    We were using Envoy Proxy
jcq3 4y ago
I thought service mesh main use case was to reduce time to production delivery, allowing hotfixes to be much more reactive. Am I totally wrong?
- jcq3 4y ago
  I think it is named Blue-green deployments in the article
- jspdown 4y ago
  It's not about fast delivery, at least not in this way. Arguably if you need mTLS, traffic shaping, cross service observability, service discovery... Then yes, it's much faster to use an existing solution than built it yourself. But it won't make you hot fixes shipped faster. Service Mesh is nothing new. People tend to call it differently back then. The key features it bring are: - Traffic shaping: mirroring, canary, blue-green... - Cross service observability - End to end encryption - Service discovery
gillh 4y ago
Service meshes make it easier to roll out advanced load management/reliability features such as prioritized load shedding, which would otherwise need to be implemented within each language/framework. For instance, Aperture[0] open-source flow control system is built on service meshes. [0]: https://github.com/fluxninja/aperture https://github.com/fluxninja/aperture [1]: https://docs.fluxninja.com https://docs.fluxninja.com

135 points by biggestlou 4y ago

asim 4y ago

Article is from 2020. Please add to title.

dang 4y ago
Added. Thanks!

samsquire 4y ago

Thanks for this. I have never deployed a service mesh or used one but I am designing something similar at the code layer. It is designed to route between server components. That is, at the architecture between threads in a multithreaded system. The problem I want to solve is that I want architecture to be trivially easy to change with minimal code changes. This is the promise and allure of enterprise service buses and messaging queues and probably Spring. I have managed RabbitMQ and I didn't enjoy it. If I want a system that can scale up and down and that multiples of any system object can be introduced or removed without drastic rewrites. I would like to decouple bottleneck from code and turn it into runtime configuration. My understanding of things such as Traefik and istio is that they are frustrating to set up. Specifically I am working on designing interthread communication patterns for multithreaded software. How do you design an architecture that is easy to change, scales and is flexible? I am thinking of a message routing definition format that is extremely flexible and allows any topology to be created. https://github.com/samsquire/ideas4#526-multiplexing-setting-format-and-routing-operation https://github.com/samsquire/ideas4#526-multiplexing-setting... I think there is application of the same pattern to the network layer too. Each communication event has associated with it an environment of keyvalues that look similar to this: petsserver1 container1 thread3 socket5 user563 ingestionthread1 These can be used to route to keyspace ranges (such as particular users to tenant shards or load balance) to other components. For example users1-1000 are handled by petsserver1 and socket5 is associated with thread3. In other words: changing the RabbitMQ routing settings doesn't change the architecture of your software. You need to change the architecture of the software to match the routing configuration. But what if you changed the routing configuration and the application architecture changed to match?

CraigJPerry 4y ago
>> But what if you changed the routing configuration and the application architecture changed to match? If there were 3 ways to categorise scaling (there’s more than this in reality) then they might be vertical, horizontal then distributed. You’re describing an architecture that’s in the horizontal scaling world view. You’re not in vertical because you’re using higher powered (but slower) strategies like active routing for comms between components where in vertical you’d have configurable queues but no routing layer. You’re not in distributed scaling mode because your routing is assuming consistent latency and consistent bandwidth behaviours. I don’t think one architecture to rule them all is a solvable problem. I’d heartily and very gratefully welcome being proven wrong on this.
- samsquire 4y ago
  Thanks for your comment. It's definitely food for thought. You remind me of fallacies of distributed computing by mentioning consistent latency and bandwidth. https://en.m.wikipedia.org/wiki/Fallacies_of_distributed_computing https://en.m.wikipedia.org/wiki/Fallacies_of_distributed_com... I'm still at the design stage. Those architectures you describe, I am hoping there is a representation that can describe many architectures. There's probably architecture's I am yet to think of that are unrepresentable with my format. Going from 1 to N or removing, adding a layer should be automatable. That's my hope anyway. I want everything to wire itself automatically. I am trying to come up with a data structure that can represent architecture. I am trying to do what inversion of control containers do per request but for architecture. In Inversion of control containers you specify a scope of an object that is instantiated for a scope such as for a request or for a session. I want that for architecture.
  - CraigJPerry 4y ago
    it’s such a fundamental problem space and with such a rich diversity of possible solutions that at a minimum you’re going to create something seriously useful for a subset of types of application. But it’d be transformational for computing if you cracked the whole problem. I hope you do. I do like your idea of outsourcing the wiring (an error prone, detail heavy task) away from humans.

NovemberWhiskey 4y ago

Now imagine you have something that has the complexity and change volume of a distributed control plane bringing together load-balancing, service advertisement, public key infrastructure, and software defined networking, and then try to imagine running it at the same reliability as your DNS. Also: proxies, proxies everywhere, as far as the eye can see.

zidad 4y ago
And in addition to that, all of those immediately becoming the same centralized single point of failure. What could possibly go wrong (on high load)? ;p
- jspdown 4y ago
  In most implementations this is not the case. Service Meshes tend to either follow a sidecar or a DaemonSet approach. You don't have a single proxy, people usually complain about the exact opposite.
darkwater 4y ago
This is the production ready part that people will usually discover later on their own skin...

Scubabear68 4y ago

I’ve only read about Service Mesh, my impression was that it seems to add an awful lot of processes and complexity just to make developer’s lives slightly easier. Maybe I’m wrong but it almost feels like busy work for DevOps. Is my first impression wrong? Is this the right way to architect systems in some use cases, and if so what are they?

MoOmer 4y ago
Many of the use cases described in the post are solved by service meshes. So, in my opinion, the questions are introspective: - “Do I have enough context to know what problem those solutions are solving, and to at least appreciate the problem space to understand why someone may solve it like this?” - “Do I have or perceive those problem to impact my infrastructure/applications?” - “Does the solution offered by the use cases described appeal to me?” If yes at the end, then one potential implementation is a service mesh. A lot of these are solved out-of-the-box with Hashicorp’s Nomad/Consul/Vault pairing, for example!
- nazka 4y ago
  I like how you frame the questions. How many times people pick a technology without answering them? Even having some knowledge in them. I am wondering does Nomad/Consul continue to scale after some level?
  - chromatin 4y ago
    I don't know about Consul, but Nomad has been scaled to 2,000,000 containers on >6000 hosts https://www.hashicorp.com/c2m https://www.hashicorp.com/c2m
- remram 4y ago
  It is true that a lot of those use cases are covered by "basic" Kubernetes (or Nomad) without the addition of Istio or similar, e.g. service discovery, load-balancing, circuit-breaking, autoscaling, blue-green, isolation, health checking... Adding a service mesh onto Kubernetes seems to bring a lot of complexity for a few benefits (80% of the effort for the last 20% sort of deal).

jpdb 4y ago

I just wrote something extremely similar, but it's only internal right now. I personally find that the service mesh value-prop is hard to justify for a serverless stack (mostly Cloud Run, but AWS Lambda too probably), and in situations where your services are mostly all in the same language and you can bake the features into libraries that are much easier to import. Observability is a great example of this. In serverless-land, you're already getting the standard HTTP metrics (ex request count, response codes, latency, etc), tracing, and standard HTTP request logging "for free."

davewritescode 4y ago
> I personally find that the service mesh value-prop is hard to justify for a serverless stack (mostly Cloud Run, but AWS Lambda too probably), and in situations where your services are mostly all in the same language and you can bake the features into libraries that are much easier to import. If you’re running server less you already have 90% of what you’d get from a service mesh. I will tell you that having seen what happens in big companies, baking distributed concerns into libraries always ends in disaster long after you’re gone. When you have a piece of code deployed in 200 separate apps, every change requires tons of project management.

4pkjai 4y ago

I really did not enjoy dealing with our service mesh at the last place I worked.

jspdown 4y ago
Out of curiosity, was it something built internally? Or were you relying on a public solution?
- 4pkjai 4y ago
  We were using Envoy Proxy

jcq3 4y ago

I thought service mesh main use case was to reduce time to production delivery, allowing hotfixes to be much more reactive. Am I totally wrong?

jcq3 4y ago
I think it is named Blue-green deployments in the article
jspdown 4y ago
It's not about fast delivery, at least not in this way. Arguably if you need mTLS, traffic shaping, cross service observability, service discovery... Then yes, it's much faster to use an existing solution than built it yourself. But it won't make you hot fixes shipped faster. Service Mesh is nothing new. People tend to call it differently back then. The key features it bring are: - Traffic shaping: mirroring, canary, blue-green... - Cross service observability - End to end encryption - Service discovery

gillh 4y ago

Service meshes make it easier to roll out advanced load management/reliability features such as prioritized load shedding, which would otherwise need to be implemented within each language/framework. For instance, Aperture[0] open-source flow control system is built on service meshes. [0]: https://github.com/fluxninja/aperture https://github.com/fluxninja/aperture [1]: https://docs.fluxninja.com https://docs.fluxninja.com