6 ms·

What happened to dumb pipes, smart endpoints? We do the same things again that we did before with SOA, having hard-to-replace middleware / bus systems.

rshriram 9y ago
It depends on how you view it. Baking everything into the application logic works if you have a homogeneous stack. When you start going down the polyglot route, with 5-6 language runtimes, multiple databases, it becomes really really hard to maintain all the communication logic (discovery, load balancing, resilience) in the app code. Take a look at AirBnB's smartstack, Yelp, Lyft's Envoy mesh. These are all polyglot applications, where the communication aspects are abstracted out into sidecars.
- einrealist 9y ago
  Maybe it depends on the size / complexity of the services. In the 'serverless' trend, such a middleware makes more sense. But essentially, moving too much logic into a middleware may support bad, lazy design decisions that make the middleware a single point of failure - like ESBs.
  - rdli 9y ago
    The middleware in this case doesn't work like an ESB in the sense that all connections are point-to-point, intermediated by a service discovery mechanism. So you're not putting all your L7 traffic onto a big bus.
  - agentgt 9y ago
    I'm a big fan of buses or perhaps more accurately I guess semi intelligent pipes with semi intelligent endpoints. I have used RabbitMQ (and Kafka at times) for large portion of my career and anytime I tell people this the bring up the single point of failure. The single point of failure arguments is getting really old and is fairly baseless particularly if your storage (e.g. RDBMS) or network (single zone or even multi zone load balancer) is also not a single point of failure. So to avoid this red herring of a problem for many who really don't need to solve that problem they create massively complicated endpoints. This endpoint code is proprietary and often has to be deployed at the same time and creates fairly tight coupling. The Netflix Hystrix creator (Ben Christensen) discussed this issue at length here: https://www.microservices.com/talks/dont-build-a-distributed-monolith/ https://www.microservices.com/talks/dont-build-a-distributed... (its also ironic that he built Hystrix and various dependencies which are sort of the antithesis of this... I guess 20/20 hindsight). There are pros and cons and its not always "lazy design decision".
    - philsnow 9y ago
      > The single point of failure arguments is getting really old and is fairly baseless particularly if your storage (e.g. RDBMS) or network (single zone or even multi zone load balancer) is also not a single point of failure. That sounds like multiple single points of failures, which just means you have more work to do, not that you can throw your hands in the air and say "welp".
    - einrealist 9y ago
      I agree. Essentially it boils down to whether communication protocols and formats are open enough and replaceable with reasonable effort. But with single point of failure, I don't mean outages. I mean, is it open and transparent enough towards your endpoints, that you can easily replace and / or fix things. Vendors go out of business or even a larger issue: your business changes.
- gshulegaard 9y ago
  Just pointing it out, but Envoy is a key piece of Istio (as I understand it).
puzzle 9y ago
Smart endpoints in N different languages means you have to do the same work (auth, rate-limiting, etc.) N times.
- venture_lol 9y ago
  Possibly. However, tooling and code generation can help. If the smarts is just a state machine, and the rules are abstracted at the right level, it would not be that hard to support. "Thick" smartpoints can save on operating cost (infrastructure, personnel, complexity, level of expertise) when your system is "simple enough" At massive scale, from an economies of scale (all things considered), it may make better sense to go with "Thin" smart endpoints. The migration between the 2 models is where the art of the design comes in play :) It's an art because it is a decision process involving many players with conflicting and competing goals, decisions, time frames, motivations etc. It's often not just a "technical" decision
  - puzzle 9y ago
    Actually, I didn't like the sidecar idea at first, given my experience with it ten years ago (we ended up changing architectures twice). But that was in a project where everything was in the language (C++) that was best supported by all the internal libraries. I've since been exposed to the vagaries of dealing with multiple languages. I think code generation can only help so much, if you care about performance and latency. Things like flow control and handling memory pressure are easier in some languages vs. others. Not going very far, if you look at gRPC, it really has three implementations: C++, Java and Go. All other supported languages wrap the first. They have different defaults, feature sets and performance characteristics. Not to mention the behavior under load. In theory it shouldn't be like that.
parasubvert 9y ago
The REST architecture always included the possibility of gateways and proxies in the end-to-end communication path to delegate shared responsibilities out of the user agent or origin server. This balances the need for centralized admin of some things and decentralized deployment of other things. Most microservices systems, even if they're not using HTTP in favor of something like gRPC, Kafka, or Rabbit, are taking a lot of WebArch lessons to heart in how they manage their policies, routes, etc. balancing centralized management over decentralized evolution. The problem with SOA-in-practice was that everything flowed through a monolithic ESB as both client and origin server, that needed to have omniscient knowledge of every route, transformation, etc., and was often a single administrative bottleneck, fault domain, etc. Some SOA frameworks had service mesh patterns where you could deploy decentralized engines with your services, but without cloud IaaS/PaaS circa 2006-2007, there was no way to maintain/deploy/upgrade these policy agents without a heavy operational burden. In sum: CORBA, COM+ or SOAP/HTTP were about mostly-centralized approaches to distributed services, REST was about extreme decentralized evolution over decades, most are looking for something akin to a dial where they can have something a bit more controlled than dozens of independent gRPC/HTTP/Rabbit/Kafka producers-consumers but not stupid like the SOAP/HTTP days. Modern cloud native service mesh approaches like this Istio thing (NetflixOSS Zuul+Eureka+Ribbon or Linkerd are alternatives) are just decentralized gateways and proxies, possibly with a console/management appliance that makes it easy to propagate changes out across a subset of your microservices. This has the benefit of allowing you to default to decentralized freedom for your various microservices but for areas where you want administrative control over policy over a set of them , you don't have to go in and tweak 15 different configs. NetflixOSS really pioneered this pattern. Netflix managed to use things like Cassandra and Zuul hot-deploy filters as the means to updated routing/health/balancing configs across their fleet of microservices. There are alternative ways to handle this of course - Hashicorp's Consul piggybacks DNS and expects your client to figure things out via their REST API or DNS queries. There are also things like RabbitMQ or a REST-polling mechanism to propagate config changes perhaps, as not everyone wants Cassandra. New frameworks like Istio or Linkerd are further alternatives. We're spoiled for choice, better or worse..
- moondev 9y ago
  Very well put. From what I see this is like moving hysterix from the sidecar level to the network level. No need for applications to care of even know about leveraging the circuit-breaking. Very excited to try it out!
venture_lol 9y ago
app <-> agent <-> hub <-> agent <-> app OR app <-> agent <-> agent <-> app OR app|agent <-> ... <-> agent|app Fundamentally, it is the same problem space. Does the apple pie taste differently with the pie in the apple vs apple in the pie? Maybe in some cases and maybe not in other cases

rshriram 9y ago

It depends on how you view it. Baking everything into the application logic works if you have a homogeneous stack. When you start going down the polyglot route, with 5-6 language runtimes, multiple databases, it becomes really really hard to maintain all the communication logic (discovery, load balancing, resilience) in the app code. Take a look at AirBnB's smartstack, Yelp, Lyft's Envoy mesh. These are all polyglot applications, where the communication aspects are abstracted out into sidecars.

einrealist 9y ago
Maybe it depends on the size / complexity of the services. In the 'serverless' trend, such a middleware makes more sense. But essentially, moving too much logic into a middleware may support bad, lazy design decisions that make the middleware a single point of failure - like ESBs.
- rdli 9y ago
  The middleware in this case doesn't work like an ESB in the sense that all connections are point-to-point, intermediated by a service discovery mechanism. So you're not putting all your L7 traffic onto a big bus.
- agentgt 9y ago
  I'm a big fan of buses or perhaps more accurately I guess semi intelligent pipes with semi intelligent endpoints. I have used RabbitMQ (and Kafka at times) for large portion of my career and anytime I tell people this the bring up the single point of failure. The single point of failure arguments is getting really old and is fairly baseless particularly if your storage (e.g. RDBMS) or network (single zone or even multi zone load balancer) is also not a single point of failure. So to avoid this red herring of a problem for many who really don't need to solve that problem they create massively complicated endpoints. This endpoint code is proprietary and often has to be deployed at the same time and creates fairly tight coupling. The Netflix Hystrix creator (Ben Christensen) discussed this issue at length here: https://www.microservices.com/talks/dont-build-a-distributed-monolith/ https://www.microservices.com/talks/dont-build-a-distributed... (its also ironic that he built Hystrix and various dependencies which are sort of the antithesis of this... I guess 20/20 hindsight). There are pros and cons and its not always "lazy design decision".
  - philsnow 9y ago
    > The single point of failure arguments is getting really old and is fairly baseless particularly if your storage (e.g. RDBMS) or network (single zone or even multi zone load balancer) is also not a single point of failure. That sounds like multiple single points of failures, which just means you have more work to do, not that you can throw your hands in the air and say "welp".
  - einrealist 9y ago
    I agree. Essentially it boils down to whether communication protocols and formats are open enough and replaceable with reasonable effort. But with single point of failure, I don't mean outages. I mean, is it open and transparent enough towards your endpoints, that you can easily replace and / or fix things. Vendors go out of business or even a larger issue: your business changes.
gshulegaard 9y ago
Just pointing it out, but Envoy is a key piece of Istio (as I understand it).

puzzle 9y ago

Smart endpoints in N different languages means you have to do the same work (auth, rate-limiting, etc.) N times.

venture_lol 9y ago
Possibly. However, tooling and code generation can help. If the smarts is just a state machine, and the rules are abstracted at the right level, it would not be that hard to support. "Thick" smartpoints can save on operating cost (infrastructure, personnel, complexity, level of expertise) when your system is "simple enough" At massive scale, from an economies of scale (all things considered), it may make better sense to go with "Thin" smart endpoints. The migration between the 2 models is where the art of the design comes in play :) It's an art because it is a decision process involving many players with conflicting and competing goals, decisions, time frames, motivations etc. It's often not just a "technical" decision
- puzzle 9y ago
  Actually, I didn't like the sidecar idea at first, given my experience with it ten years ago (we ended up changing architectures twice). But that was in a project where everything was in the language (C++) that was best supported by all the internal libraries. I've since been exposed to the vagaries of dealing with multiple languages. I think code generation can only help so much, if you care about performance and latency. Things like flow control and handling memory pressure are easier in some languages vs. others. Not going very far, if you look at gRPC, it really has three implementations: C++, Java and Go. All other supported languages wrap the first. They have different defaults, feature sets and performance characteristics. Not to mention the behavior under load. In theory it shouldn't be like that.

parasubvert 9y ago

The REST architecture always included the possibility of gateways and proxies in the end-to-end communication path to delegate shared responsibilities out of the user agent or origin server. This balances the need for centralized admin of some things and decentralized deployment of other things. Most microservices systems, even if they're not using HTTP in favor of something like gRPC, Kafka, or Rabbit, are taking a lot of WebArch lessons to heart in how they manage their policies, routes, etc. balancing centralized management over decentralized evolution. The problem with SOA-in-practice was that everything flowed through a monolithic ESB as both client and origin server, that needed to have omniscient knowledge of every route, transformation, etc., and was often a single administrative bottleneck, fault domain, etc. Some SOA frameworks had service mesh patterns where you could deploy decentralized engines with your services, but without cloud IaaS/PaaS circa 2006-2007, there was no way to maintain/deploy/upgrade these policy agents without a heavy operational burden. In sum: CORBA, COM+ or SOAP/HTTP were about mostly-centralized approaches to distributed services, REST was about extreme decentralized evolution over decades, most are looking for something akin to a dial where they can have something a bit more controlled than dozens of independent gRPC/HTTP/Rabbit/Kafka producers-consumers but not stupid like the SOAP/HTTP days. Modern cloud native service mesh approaches like this Istio thing (NetflixOSS Zuul+Eureka+Ribbon or Linkerd are alternatives) are just decentralized gateways and proxies, possibly with a console/management appliance that makes it easy to propagate changes out across a subset of your microservices. This has the benefit of allowing you to default to decentralized freedom for your various microservices but for areas where you want administrative control over policy over a set of them , you don't have to go in and tweak 15 different configs. NetflixOSS really pioneered this pattern. Netflix managed to use things like Cassandra and Zuul hot-deploy filters as the means to updated routing/health/balancing configs across their fleet of microservices. There are alternative ways to handle this of course - Hashicorp's Consul piggybacks DNS and expects your client to figure things out via their REST API or DNS queries. There are also things like RabbitMQ or a REST-polling mechanism to propagate config changes perhaps, as not everyone wants Cassandra. New frameworks like Istio or Linkerd are further alternatives. We're spoiled for choice, better or worse..

moondev 9y ago
Very well put. From what I see this is like moving hysterix from the sidecar level to the network level. No need for applications to care of even know about leveraging the circuit-breaking. Very excited to try it out!

venture_lol 9y ago

app <-> agent <-> hub <-> agent <-> app OR app <-> agent <-> agent <-> app OR app|agent <-> ... <-> agent|app Fundamentally, it is the same problem space. Does the apple pie taste differently with the pie in the apple vs apple in the pie? Maybe in some cases and maybe not in other cases