3 ms·

I understand what this pattern does, but it makes me wonder how useful it is in practice. I am focusing specifically on the retry and circuit breaker functional

I understand what this pattern does, but it makes me wonder how useful it is in practice. I am focusing specifically on the retry and circuit breaker functionality. If the call is done in a synchronous context (for example original client executes a REST call), you are adding latency by inserting an extra service in between. When the remote service is not immediately available or responsive or fails, and you do a retry, in the end your call might take too long and the caller might cancel the request. I have seen this behavior in practice and it makes me wonder how useful this implementation is in a synchronous context. You add complexity (on the infra level) and you add latency in the happy path. When a retry is needed, most of the time (in my experience) the call to the remote service does not succeed in time, and the original call still fails. If the call is done in an asynchronous context (for example the original client picks up a message from a queue and processes it), you are adding latency and complexity by inserting an extra service and extra logic. However, when the remote service is not immediately available, you can just let the processing of the message fail. The queue or bus should contain retry logic that could be finetuned based on the type of error you get. So, in that case, you should already have a retry mechanism out of the box, then is the added latency and complexity really worth it? I understand about the circuit breaker, and I understand that might be useful, because it could prevent the remote service being overloaded with requests (well, at least if every caller to the remote service implements circuit breaker... but then, the ambassador service would better be placed on the side of the remote service and every client should be forced to pass through it, and then it might just be implemented inside the remote service instead of in a separate service adding extra latency/complexity/... basically, the remote service should protect itself against this). Thoughts? Does it make sense what I am thinking?

somerando7 3y ago
It's pretty useful when you have some distribution layer (i.e. some pubsub system) Consider 10-15 applications running on a host, and all of them are listening to data being distributed by another service. Instead of all of them opening a connection to that service, instead they would all be connected to this sidecar, and the sidecar would merge the distribution of data (and subscriptions) to the pubsub system
ablob 3y ago
You seem to be quite focused on latency. Latency is not necessarily something you care about. For coarse grained logistics information, for example, it doesn't really matter if the push notification arrives a few tens or hundreds of milliseconds (or even seconds) later. So if you don't like the latency aspects I am inclined to ask: "What do you need the low latency for?". The explanation on the page itself states the following: Use this pattern when you: * Need to build a common set of client connectivity features for multiple languages or frameworks. * Need to offload cross-cutting client connectivity concerns to infrastructure developers or other more specialized teams. * Need to support cloud or cluster connectivity requirements in a legacy application or an application that is difficult to modify. This pattern may not be suitable: * When network request latency is critical. * When client connectivity features are consumed by a single language. * When connectivity features cannot be generalized and require deeper integration with the client application. If you want to question the usefulness of the pattern, it is best to argue against the use-cases in which it is recommended to be used instead of using a scenario the pattern _explicitly_ states it is not meant for (i.e.: low latency). Now you also had some thoughts on complexity. Regarding what you said: (retries, resiliency, extra logic) it will have to live somewhere. You don't add meaningless complexity for the sake of it after all. How are you going to add another retry policy to a blob you have no control over otherwise? Shifting the burden of networking is also an explicit option listed in the "suitable for" section. You can decide where complexity lives after all. If accidental complexity is your issue I am inclined to ask where you see it here in general. Both the ambassador pattern and your proposed alternatives add overhead in terms of complexity somewhere. I'm struggling to see a clear favorite here. Explicitly regarding your last statement that "[...] the remote service should protect itself against this).": Retries and Monitoring are things the remote service can't do by itself by definition. Even load balancing/shedding might not be solvable by it depending on the situation. Notice that circuit breaking is not the only thing the ambassador is used for. Network related configuration updates (see section "Context and problem") are something that might not be done by the remote service either.
- rvdginste 3y ago
  What I had in mind, is a webapp that does backend api calls. That is a synchronous call, in the sense that often the user is waiting for it since it is often the result of an interaction with the website. I did not consider that a low latency requirement. Still, when a backend service (accessed either directly or indirectly) is not available or has problems, and there is a retry mechanism, this quickly runs into seconds. > You don't add meaningless complexity for the sake of it after all. > You can decide where complexity lives after all. Good points and something I should think about when designing systems. You have good and interesting points, and it is true that I am very wary about introducing extra latency in the context of an http api that is used by a webapp. With the infra that is available today, it is possible to build snappy webapps, but my feeling is that you have to be wary about introducing extra "hops" in the execution of one http call, even though that is not strictly a "low latency" requirement.

somerando7 3y ago

It's pretty useful when you have some distribution layer (i.e. some pubsub system) Consider 10-15 applications running on a host, and all of them are listening to data being distributed by another service. Instead of all of them opening a connection to that service, instead they would all be connected to this sidecar, and the sidecar would merge the distribution of data (and subscriptions) to the pubsub system

ablob 3y ago

You seem to be quite focused on latency. Latency is not necessarily something you care about. For coarse grained logistics information, for example, it doesn't really matter if the push notification arrives a few tens or hundreds of milliseconds (or even seconds) later. So if you don't like the latency aspects I am inclined to ask: "What do you need the low latency for?". The explanation on the page itself states the following: Use this pattern when you: * Need to build a common set of client connectivity features for multiple languages or frameworks. * Need to offload cross-cutting client connectivity concerns to infrastructure developers or other more specialized teams. * Need to support cloud or cluster connectivity requirements in a legacy application or an application that is difficult to modify. This pattern may not be suitable: * When network request latency is critical. * When client connectivity features are consumed by a single language. * When connectivity features cannot be generalized and require deeper integration with the client application. If you want to question the usefulness of the pattern, it is best to argue against the use-cases in which it is recommended to be used instead of using a scenario the pattern _explicitly_ states it is not meant for (i.e.: low latency). Now you also had some thoughts on complexity. Regarding what you said: (retries, resiliency, extra logic) it will have to live somewhere. You don't add meaningless complexity for the sake of it after all. How are you going to add another retry policy to a blob you have no control over otherwise? Shifting the burden of networking is also an explicit option listed in the "suitable for" section. You can decide where complexity lives after all. If accidental complexity is your issue I am inclined to ask where you see it here in general. Both the ambassador pattern and your proposed alternatives add overhead in terms of complexity somewhere. I'm struggling to see a clear favorite here. Explicitly regarding your last statement that "[...] the remote service should protect itself against this).": Retries and Monitoring are things the remote service can't do by itself by definition. Even load balancing/shedding might not be solvable by it depending on the situation. Notice that circuit breaking is not the only thing the ambassador is used for. Network related configuration updates (see section "Context and problem") are something that might not be done by the remote service either.

rvdginste 3y ago
What I had in mind, is a webapp that does backend api calls. That is a synchronous call, in the sense that often the user is waiting for it since it is often the result of an interaction with the website. I did not consider that a low latency requirement. Still, when a backend service (accessed either directly or indirectly) is not available or has problems, and there is a retry mechanism, this quickly runs into seconds. > You don't add meaningless complexity for the sake of it after all. > You can decide where complexity lives after all. Good points and something I should think about when designing systems. You have good and interesting points, and it is true that I am very wary about introducing extra latency in the context of an http api that is used by a webapp. With the infra that is available today, it is possible to build snappy webapps, but my feeling is that you have to be wary about introducing extra "hops" in the execution of one http call, even though that is not strictly a "low latency" requirement.