5 ms·

Congrats on the release, looks like a great platform for my use case (an app with a series of microservices, each of which needs to scale independently and for

Congrats on the release, looks like a great platform for my use case (an app with a series of microservices, each of which needs to scale independently and for which requests are usually long running). I do have a question re: Flynn's router. The router README [1] states that it uses a random load balancing algorithm. I worry that at scale it would exhibit issues similar to what Heroku's "routing mesh" has (had?) as explained in [2], namely that some nodes would sit idle while others simultaneously let requests build up in their queue, hurting overall latency and reducing the marginal value of each additional node. Any chance of something like HAProxy's leastconn algorithm landing in Flynn's router to mitigate this type of worry? [1] https://github.com/flynn/flynn/tree/master/router https://github.com/flynn/flynn/tree/master/router [2] http://genius.com/James-somers-herokus-ugly-secret-annotated http://genius.com/James-somers-herokus-ugly-secret-annotated

Titanous 11y ago
Great question! We're aware of the Heroku issues, and they don't fully apply in our case. We currently do not have any queues in the router other than the kernel's accept queue. All incoming connections are assigned a goroutine immediately and the requests are proxied back to service instances without queuing. So if your application instances can handle concurrent requests, the performance impact of random load balancing should not be huge. We are absolutely open to implementing other balancing algorithms, including least connections. The one caveat is that at any time multiple routers will be in operation and we're not going to do extensive coordination between them, so the balancing will never be perfect.
- mikejarema 11y ago
  Got it, makes complete sense. Thanks for the direct answer. A bit more info on the app I'm thinking of testing with Flynn: each instance has a single dedicated resource (actually a psuedo terminal sitting on top of a command line tool). Any given instance is incapable of handling requests concurrently, they queue up at the instance. So the app's concurrency model relies on smart load balancing at the router itself. I've got HAProxy playing this role, but have done a bit of a hack job juggling the containers which implement the various microservices. I'll give Flynn a go to see if the simplicity it gives me in managing my deployment is worth the routing tradeoff for now, but I would definitely love to see Flynn's router adapted as you describe.

Titanous 11y ago

Great question! We're aware of the Heroku issues, and they don't fully apply in our case. We currently do not have any queues in the router other than the kernel's accept queue. All incoming connections are assigned a goroutine immediately and the requests are proxied back to service instances without queuing. So if your application instances can handle concurrent requests, the performance impact of random load balancing should not be huge. We are absolutely open to implementing other balancing algorithms, including least connections. The one caveat is that at any time multiple routers will be in operation and we're not going to do extensive coordination between them, so the balancing will never be perfect.

mikejarema 11y ago
Got it, makes complete sense. Thanks for the direct answer. A bit more info on the app I'm thinking of testing with Flynn: each instance has a single dedicated resource (actually a psuedo terminal sitting on top of a command line tool). Any given instance is incapable of handling requests concurrently, they queue up at the instance. So the app's concurrency model relies on smart load balancing at the router itself. I've got HAProxy playing this role, but have done a bit of a hack job juggling the containers which implement the various microservices. I'll give Flynn a go to see if the simplicity it gives me in managing my deployment is worth the routing tradeoff for now, but I would definitely love to see Flynn's router adapted as you describe.