11 ms·

Microservices are great, but the underlying protocol they run on is important. If you're building a REST interface to all your services, and something consumes

by viksit 11y ago

Microservices are great, but the underlying protocol they run on is important. If you're building a REST interface to all your services, and something consumes them - they might be slower than a monolithic app unless you have something like a TCP or HTTP level keep-alive built in. Connections need to be long standing - otherwise the overhead of creating a new connection is pretty high. Question here - what is a good way to make this long standing connection happen? Eg, if you use python urllib3 and nginx - can you keep these connections alive enough (with pings or whatever) that your latency is lower than bundling that service within the code itself as a library?

usaar333 11y ago
If you use keep alive and your server/load balancer doesn't have overly aggressive connection termination policies, I doubt re-connections every 60 or so s will be a major throughput hit. Regardless, you are always going to incur latency overhead by sending data over TCP relative to just sharing memory. What I am more worried about with microservices is the data serialization overhead. Transforming data to some encoding that is robust against version changes (say using protobuf) can be quite costly on both sender and receiver, especially in languages with relatively slow object creation (e.g. python). This is highly application specific, but I'd love to hear others' thoughts on this trade-off.
- viksit 11y ago
  Using a binary serialized format is actually pretty good that way. For a high throughput API that I built, shifting to protobuf from json, and using binary formats made a lot of difference. The packet size is lower, you can use JVM based platforms to do analysis and analytics (esp when storing logs as proto) - those benefits outweigh the slowness of the object creation (which isn't THAT slow on cpython). Atleast on the proto 2.3.x versions that I use. That said, to my point of keep alives - lets say you've got process A talking to process B on localhost, which is making web service calls to the internet. Every 5ms delay is hurting your total response time, especially when your connection pool is waiting on a dropped connection to be reinstated.

by viksit 11y ago

usaar333 11y ago

If you use keep alive and your server/load balancer doesn't have overly aggressive connection termination policies, I doubt re-connections every 60 or so s will be a major throughput hit. Regardless, you are always going to incur latency overhead by sending data over TCP relative to just sharing memory. What I am more worried about with microservices is the data serialization overhead. Transforming data to some encoding that is robust against version changes (say using protobuf) can be quite costly on both sender and receiver, especially in languages with relatively slow object creation (e.g. python). This is highly application specific, but I'd love to hear others' thoughts on this trade-off.

viksit 11y ago
Using a binary serialized format is actually pretty good that way. For a high throughput API that I built, shifting to protobuf from json, and using binary formats made a lot of difference. The packet size is lower, you can use JVM based platforms to do analysis and analytics (esp when storing logs as proto) - those benefits outweigh the slowness of the object creation (which isn't THAT slow on cpython). Atleast on the proto 2.3.x versions that I use. That said, to my point of keep alives - lets say you've got process A talking to process B on localhost, which is making web service calls to the internet. Every 5ms delay is hurting your total response time, especially when your connection pool is waiting on a dropped connection to be reinstated.