7 ms·

Has anyone got an opinion on what makes a service mesh better than using a message broker for most distributed systems? Is it performance? Is it that http has b

Has anyone got an opinion on what makes a service mesh better than using a message broker for most distributed systems? Is it performance? Is it that http has become the lowest common denominator and people just don't want to use any other communication protocol?

pmlnr 7y ago
It's only better when you have a MASSIVE system and the chances of the message broker infrastructure not being capable enough is a problem.
jordanbeiber 7y ago
My take. Might be clueless though, but in my experience: The protocol has less to do with it, see grpc for example. Having run an operationally critical microservice infrastructure with consul for service discovery I realized that a mesh makes sense. When you start hitting a few hundreds of services you’ll want: 1. The observability. Scenario: Why is the application slow? The application depends on two different services each with individual sets of downstream dependencies. 2. Making the services discovery process more resilient. An outage of consul without other means of discovery sucks! Message brokering as well as event sourcing seems to this layman like awesome patterns where eventual consistency is perfectly ok. In most scenarios it probably is, but for some it is not. Whatever works!
- fogetti 7y ago
  I don't really see how the service mesh solves eventual consistency. You might call a service which handles all the tasks asynchronously similarly to a message in a message broker system. Service discovery is also inherent in the routing rules itself when you use something like AMQP for example. Observability might be something that the broker system lacks of I agree.
  - jordanbeiber 7y ago
    Oh no, it doesn’t and it wasn’t what I wanted to imply. But you might have scenarios where you want or require synchronous or continous direct service to service communication is what I meant.
jatins 7y ago
Message brokers only work for "fire-and-forget" kind of async operations. As in, when you make a request you are not waiting for a response. I don't think it is service mesh OR brokers. Both cater to different use cases.
- discreteevent 7y ago
  This is true but I would think that if you had to pick one then it is much easier to make a message broker do reliable request-reply/pull than service mesh do reliable events/push.
  - sagichmal 7y ago
    The opposite tends to be true. https://programmingisterrible.com/post/162346490883/how-do-you-cut-a-monolith-in-half/ https://programmingisterrible.com/post/162346490883/how-do-y...
    - discreteevent 7y ago
      Good article, thanks: "A protocol is the rules and expectations of participants in a system, and how they are beholden to each other. A protocol defines who takes responsibility for failure. The problem with message brokers, and queues, is that no-one does. Using a message broker is not the end of the world, nor a sign of poor engineering. Using a message broker is a tradeoff. Use them freely knowing they work well on the edges of your system as buffers. Use them wisely knowing that the buck has to stop somewhere else. Use them cheekily to get something working. I say don’t rely on a message broker, but I can’t point to easy off-the-shelf answers. HTTP and DNS are remarkable protocols, but I still have no good answers for service discovery." I would not disagree with any of this but I don't know enough about service meshes to know if they mean that you don't need a protocol (meaning the edges still need to deal with failures after the mesh or broker has retried etc). One thing a broker gives you is that if the problem is that the other service is busy or down then it will eventually get the message. So it avoids one specific kind of failure in the case where eventual consistency is acceptable.
- 7y ago
scraegg 7y ago
If you see an airplane that flies you might conclude airplanes are a good transportation pattern. But if you see there's a boat, that promises to fly, and you see that people taped a lot of helicopters onto it to make it fly, then probably you would conclude that boats are bad at flying. Message Brokers are the airplanes. Even if you can't understand how they work you will see that many companies use them who really rely on distributed systems working, eg telephone companies, banks, trading companies. Service Meshes try to solve a problem that K8s was meant to solve in the first place. So K8s is the boat and Service Mesh is the attached helicopter. By itself it might be a good idea to use it, but the way things are taped together right now it's just an anti-pattern. If you don't have a pointy-haired boss forcing you to use it, then it's probably better to avoid the whole thing. I'd rather see how far the development around k8s-less pods with podman will go and take care of the distributed architecture of my systems myself.
- geggam 7y ago
  These layers of abstraction are nice but they seem to ignore the fact you still have to manage the systems underneath and the networking around it. At some point folks need to realize they arent google and they really dont need to abstract the development layer because they dont have a team of google SRE to manage what their pods / containers / mesh runs on
- solipsism 7y ago
  This is a strange answer. A message broker is not in the same category as Istio. Istio is for effecting policy changes (regarding security, traffic routing, monitoring, etc) without updating your code. Which is especially useful if your code is a bunch of diverse microservices. Message brokers are... not that. These are apples and elephants you're comparing.
- pm90 7y ago
  Kubernetes is a platform, it lets you do whatever the f you want to. Service meshes are one thing to build on top of the platform.
  - scraegg 7y ago

pmlnr 7y ago

It's only better when you have a MASSIVE system and the chances of the message broker infrastructure not being capable enough is a problem.

jordanbeiber 7y ago

My take. Might be clueless though, but in my experience: The protocol has less to do with it, see grpc for example. Having run an operationally critical microservice infrastructure with consul for service discovery I realized that a mesh makes sense. When you start hitting a few hundreds of services you’ll want: 1. The observability. Scenario: Why is the application slow? The application depends on two different services each with individual sets of downstream dependencies. 2. Making the services discovery process more resilient. An outage of consul without other means of discovery sucks! Message brokering as well as event sourcing seems to this layman like awesome patterns where eventual consistency is perfectly ok. In most scenarios it probably is, but for some it is not. Whatever works!

fogetti 7y ago
I don't really see how the service mesh solves eventual consistency. You might call a service which handles all the tasks asynchronously similarly to a message in a message broker system. Service discovery is also inherent in the routing rules itself when you use something like AMQP for example. Observability might be something that the broker system lacks of I agree.
- jordanbeiber 7y ago
  Oh no, it doesn’t and it wasn’t what I wanted to imply. But you might have scenarios where you want or require synchronous or continous direct service to service communication is what I meant.

jatins 7y ago

Message brokers only work for "fire-and-forget" kind of async operations. As in, when you make a request you are not waiting for a response. I don't think it is service mesh OR brokers. Both cater to different use cases.

discreteevent 7y ago
This is true but I would think that if you had to pick one then it is much easier to make a message broker do reliable request-reply/pull than service mesh do reliable events/push.
- sagichmal 7y ago
  The opposite tends to be true. https://programmingisterrible.com/post/162346490883/how-do-you-cut-a-monolith-in-half/ https://programmingisterrible.com/post/162346490883/how-do-y...
  - discreteevent 7y ago
    Good article, thanks: "A protocol is the rules and expectations of participants in a system, and how they are beholden to each other. A protocol defines who takes responsibility for failure. The problem with message brokers, and queues, is that no-one does. Using a message broker is not the end of the world, nor a sign of poor engineering. Using a message broker is a tradeoff. Use them freely knowing they work well on the edges of your system as buffers. Use them wisely knowing that the buck has to stop somewhere else. Use them cheekily to get something working. I say don’t rely on a message broker, but I can’t point to easy off-the-shelf answers. HTTP and DNS are remarkable protocols, but I still have no good answers for service discovery." I would not disagree with any of this but I don't know enough about service meshes to know if they mean that you don't need a protocol (meaning the edges still need to deal with failures after the mesh or broker has retried etc). One thing a broker gives you is that if the problem is that the other service is busy or down then it will eventually get the message. So it avoids one specific kind of failure in the case where eventual consistency is acceptable.
7y ago

scraegg 7y ago

If you see an airplane that flies you might conclude airplanes are a good transportation pattern. But if you see there's a boat, that promises to fly, and you see that people taped a lot of helicopters onto it to make it fly, then probably you would conclude that boats are bad at flying. Message Brokers are the airplanes. Even if you can't understand how they work you will see that many companies use them who really rely on distributed systems working, eg telephone companies, banks, trading companies. Service Meshes try to solve a problem that K8s was meant to solve in the first place. So K8s is the boat and Service Mesh is the attached helicopter. By itself it might be a good idea to use it, but the way things are taped together right now it's just an anti-pattern. If you don't have a pointy-haired boss forcing you to use it, then it's probably better to avoid the whole thing. I'd rather see how far the development around k8s-less pods with podman will go and take care of the distributed architecture of my systems myself.

geggam 7y ago
These layers of abstraction are nice but they seem to ignore the fact you still have to manage the systems underneath and the networking around it. At some point folks need to realize they arent google and they really dont need to abstract the development layer because they dont have a team of google SRE to manage what their pods / containers / mesh runs on
solipsism 7y ago
This is a strange answer. A message broker is not in the same category as Istio. Istio is for effecting policy changes (regarding security, traffic routing, monitoring, etc) without updating your code. Which is especially useful if your code is a bunch of diverse microservices. Message brokers are... not that. These are apples and elephants you're comparing.
pm90 7y ago
Kubernetes is a platform, it lets you do whatever the f you want to. Service meshes are one thing to build on top of the platform.
- scraegg 7y ago