2 ms·

If there's anyone from Lyft here answering questions, I have a few. #1: cost. Doesn't it basically cost double to move the request from the client application

If there's anyone from Lyft here answering questions, I have a few. #1: cost. Doesn't it basically cost double to move the request from the client application to the proxy, and then from the proxy to the backend? #2: upgrades. What happens to the clients when the proxy is being rolled out? #3: head-of-line blocking. If an application has two streams to the same backend, one stream which is low priority and one which is higher, how does the proxy handle that?

igor47 10y ago
i'm not at lyft, but i've been looking into introducing this proxy in our environment to replace haproxy in smartstack (http://nerds.airbnb.com/smartstack-service-discovery-cloud/ http://nerds.airbnb.com/smartstack-service-discovery-cloud/) and so have thought about these questions a bit. #1: the cost of local proxying is very small, on the order of tenths of ms. it's going to be hard to distinguish against the backdrop of the broader network latency. also, in our environment we've been thinking of only running the proxy on the client end for now, to make the migration easier. #2: as with haproxy, you leave the existing connections handled with the running proxy, and new connections are taken up by the new proxy. unlike haproxy, envoy actually supports live config reloading, so you don't have to pay that penalty on every config change #3: multiple connections are multiplexed, but i'm not sure how you specify priority to the proxy. do the docs specify a mechanism? i imagine both requests are served in parallel, in no particular order, and the data is buffered for the client.
mattklein123 10y ago
Hi, I work at Lyft. To answer your questions: 1) There is added cost, though it varies depending on how many things Envoy is configured to do (e.g., logging, tracing, stats, rate limiting, health checking, etc.). Even in complex scenarios (Envoy being used to proxy both inbound connections into a service, as well as proxy outbound connections to Mongo or Dynamo), we measure Envoy overhead to be < 1ms, which for almost all applications is negligible. There are definitely certain cases where this might be prohibitive, but in general we find the common functionality we get (again stats, tracing, etc.) to be invaluable in a production setting. 2) Envoy supports hot restart (https://lyft.github.io/envoy/docs/intro/arch_overview/hot_restart.html https://lyft.github.io/envoy/docs/intro/arch_overview/hot_re...), as well as graceful drain of existing connections, so there is limited/no impact to existing clients. There is one enhancement that we would like to make to our HTTP/2 graceful draining to make it even more seamless, but that is more complicated than I can type here. :) 3) Right now Envoy does not support HTTP/2 priority so all streams are treated equally. We are currently working on priority support at the routing layer, with different connection pools available for high and low priority traffic, as well as circuit breaking settings. In the future we will likely merge this back into a single HTTP/2 connection with proper priority support. In practice though, within the DC, head of line blocking at the TCP layer isn't too much of an issue.

igor47 10y ago

i'm not at lyft, but i've been looking into introducing this proxy in our environment to replace haproxy in smartstack (http://nerds.airbnb.com/smartstack-service-discovery-cloud/ http://nerds.airbnb.com/smartstack-service-discovery-cloud/) and so have thought about these questions a bit. #1: the cost of local proxying is very small, on the order of tenths of ms. it's going to be hard to distinguish against the backdrop of the broader network latency. also, in our environment we've been thinking of only running the proxy on the client end for now, to make the migration easier. #2: as with haproxy, you leave the existing connections handled with the running proxy, and new connections are taken up by the new proxy. unlike haproxy, envoy actually supports live config reloading, so you don't have to pay that penalty on every config change #3: multiple connections are multiplexed, but i'm not sure how you specify priority to the proxy. do the docs specify a mechanism? i imagine both requests are served in parallel, in no particular order, and the data is buffered for the client.

mattklein123 10y ago

Hi, I work at Lyft. To answer your questions: 1) There is added cost, though it varies depending on how many things Envoy is configured to do (e.g., logging, tracing, stats, rate limiting, health checking, etc.). Even in complex scenarios (Envoy being used to proxy both inbound connections into a service, as well as proxy outbound connections to Mongo or Dynamo), we measure Envoy overhead to be < 1ms, which for almost all applications is negligible. There are definitely certain cases where this might be prohibitive, but in general we find the common functionality we get (again stats, tracing, etc.) to be invaluable in a production setting. 2) Envoy supports hot restart (https://lyft.github.io/envoy/docs/intro/arch_overview/hot_restart.html https://lyft.github.io/envoy/docs/intro/arch_overview/hot_re...), as well as graceful drain of existing connections, so there is limited/no impact to existing clients. There is one enhancement that we would like to make to our HTTP/2 graceful draining to make it even more seamless, but that is more complicated than I can type here. :) 3) Right now Envoy does not support HTTP/2 priority so all streams are treated equally. We are currently working on priority support at the routing layer, with different connection pools available for high and low priority traffic, as well as circuit breaking settings. In the future we will likely merge this back into a single HTTP/2 connection with proper priority support. In practice though, within the DC, head of line blocking at the TCP layer isn't too much of an issue.