3 ms·

I think the explanation for the change is a bit simpler than that. Someone can correct me if I've misunderstood, but AFAIK it isn't that "Intelligent Routing" w

by chc 14y ago

I think the explanation for the change is a bit simpler than that. Someone can correct me if I've misunderstood, but AFAIK it isn't that "Intelligent Routing" was too expensive, but that it depended on simplifying assumptions that stopped being true. Originally, Heroku was only for Ruby, and it depended on the assumption that a server could only handle one request at a time. All the talk of Intelligent Routing seems to date from this time, so it appears that Intelligent Routing just meant they never routed more than one request at a time to a server. But then Heroku wanted to add support for things like Java and Node.js, which can support multiple requests per server. This meant the simplifying assumption of "1 dyno = 1 request" baked into the old routing algorithm was no longer valid, so they had to switch to something else or they'd be crippling pretty much everything but Ruby.

ChuckMcM 14y ago
That would make sense, if Heroku didn't know which were ruby requests and which weren't. But it seems like they did. If only as a set of VIPs (virtual IPs) landing on the router being tagged as 'for ruby' or 'not ruby' which could pick the appropriate routing algorithm. Understand that keeping millisecond accurate state on 10 machines is doable, on 100 machines its hard, and on a few thousand machines? It really starts to break down. One way I've seen that done is that on ingress a request is wrapped in a message for server X which is taken out of the 'free' pool, and then when server X returns the answer back through the router it gets added back into the free pool. But the next order effect is that list insertion / removal has different sorts of behaviors, if you shift frees into the end of the list and pop them from the front (a round robin approach) you get good distribution but sometimes send things 'far' away when they could be served locally. If you push/pop things from the front you get some really hot servers and some really cold servers. Early on Google played some games which were designed to maximize the use of available network backbone bandwidth (its always oversubscribed from the server to the 'net'). Like any of the more interesting problems it starts off easy and then gets harder and harder.

by chc 14y ago

ChuckMcM 14y ago

That would make sense, if Heroku didn't know which were ruby requests and which weren't. But it seems like they did. If only as a set of VIPs (virtual IPs) landing on the router being tagged as 'for ruby' or 'not ruby' which could pick the appropriate routing algorithm. Understand that keeping millisecond accurate state on 10 machines is doable, on 100 machines its hard, and on a few thousand machines? It really starts to break down. One way I've seen that done is that on ingress a request is wrapped in a message for server X which is taken out of the 'free' pool, and then when server X returns the answer back through the router it gets added back into the free pool. But the next order effect is that list insertion / removal has different sorts of behaviors, if you shift frees into the end of the list and pop them from the front (a round robin approach) you get good distribution but sometimes send things 'far' away when they could be served locally. If you push/pop things from the front you get some really hot servers and some really cold servers. Early on Google played some games which were designed to maximize the use of available network backbone bandwidth (its always oversubscribed from the server to the 'net'). Like any of the more interesting problems it starts off easy and then gets harder and harder.