5 ms·
This should be more prominent. I want to love Heroku, and am sure that I could. But really, throwing in the towel at intelligent routing and replacing it with
by FireBeyond 14y ago
This should be more prominent. I want to love Heroku, and am sure that I could.
But really, throwing in the towel at intelligent routing and replacing it with "random routing" is horrific, if true.
It's arguable that the routing mesh and scaling dynamics of Heroku are a large part, if not -the- defining reason for someone to choose Heroku over AWS directly.
Is it a "hard" problem? I'm absolutely sure it is. That's one reason customers are throwing money at you to solve it, Heroku.
- timmaah 14y agoEven their own docs were wrong on this for a long time. It bit me in the ass back in 2011 and I got them to clarify and update the documentation just a little. http://tiwatson.com/blog/2011-2-17-heroku-no-longer-using-a-global-request-queue http://tiwatson.com/blog/2011-2-17-heroku-no-longer-using-a-...
- michaelrkn 14y agoThanks for the blog post, by the way. When we were struggling with our own Heroku scaling issues last year (we eventually moved to AWS), I came across it and it was good vindication that somebody else was facing the same issue.
- chc 14y ago> But really, throwing in the towel at intelligent routing and replacing it with "random routing" is horrific, if true. The thing is, their old "intelligent routing" was really just "we will only route one request at a time to a dyno." In other words, what changed is that they now allow dynos to serve multiple requests at a time. When you put it that way, it doesn't sound as horrific, does it?
- arcatek 14y agoThe requests will not be served at the same time, that's the whole point. If a request is routed to a busy dyno, you will have to wait that the previous job finish before being able to start yours.
- damian2000 14y agoI believe the article is saying that this wait will only occur with Rails due to it being single threaded? Reactor pattern frameworks like node don't suffer the same issue.
- teraflop 14y agoThey will if the request is CPU-bound. In that case, throwing more concurrent requests at a server than it has cores just slows all of them down.
- joseph_cooney 14y agoHow many web requests are typically CPU bound though? Not many.
- vidarh 14y agoDoesn't matter. At some point you hit memory limits, disk IO limits, or simply a connection limit. It doesn't matter what limit: If you have some requests that are longer running than others, random load balancing will make them start piling up once you reach some traffic threshold. You can increase the threshold by adding more backends or increasing the capacity of each backend (by optimizing, or picking beefier hardware if you're on a platform that will let you), and maybe you can increase it enough that it won't affect you. But no matter what you do, you end up having to allocate more spare resources at it than what you would need with more intelligent routing. If you're lucky, the effect might be small enough to not cost you much money, and you might be able to ignore it, but it's still there.
- chc 14y ago> But no matter what you do, you end up having to allocate more spare resources at it than what you would need with more intelligent routing. I think we have to remember that the "intelligent routing" in question here is actually marketing-speak for "one request per server." Are you saying that when your servers can only receive one request at a time, you will necessarily need fewer than if your servers can handle three requests at a time but are assigned requests randomly?
- Dylan16807 14y agoYes it does sound as horrific. It should never route more than N requests to a dyno, where N is the number of requests it can handle simultaneously. It doesn't matter if N is 1, 2, or 10. And it should always send new requests to the emptiest dyno.
- chc 14y agoWhile I agree that routing to the emptiest dyno would be a good thing, I think it's a little melodramatic to describe giving every dyno at least three times and up to several thousand times as much power as "horrific."
- Dylan16807 14y agoPower, what? The problem is when you send a dyno that has all its threads stuck on long-running computations a new request, because it won't be able to even start processing it. The power is orthogonal to the problem. The only mitigation is that if a dyno can handle a large number of threads, it probably won't get clogged. But if it can only handle 3 and gets new requests at random, you're in a bad place.
- badgar 14y ago> That's one reason customers are throwing money at you to solve it, Heroku. People are throwing money at Heroku because it's really easy to use, not because it's the best long-term technology choice. Seriously - what percentage of Heroku paying users do you think actually read up on the finest technical details like routing algorithms before they put in their credit card? Heroku knows. They know you can't even build a highly-available service on top of it, since it's singly-homed, and they're still making tons of money.
- aneth4 14y agoSo heroku doesn't want these $20,000/mo accounts, just technically understaffed startups paying $1000/mo? I think heroku does want to be a long-term technology choice.
- badgar 14y ago> So heroku doesn't want these $20,000/mo accounts, just technically understaffed startups paying $1000/mo? > I think heroku does want to be a long-term technology choice. Oh, I'm sure Heroku wants to be a long-term technology choice. That doesn't mean they're trying to be one with their current product offerings. Consider their product choices since launch: they've added dozens of small value-added technology integrations. Features for a few bucks a month like New Relic to upsell their smallest customers. The price drop was also a big move to reduce barriers to using their platform - which also targets smaller customers. They launched Clojure as their third supported language! Meanwhile, they're singly-homed and have had several protracted outages, and have no announced plans to build a multi-homed product. Scalability has gotten worse with this random routing development. I think Heroku has known for a long time that they don't have a long-term platform product and that they can't keep big accounts until they build one.
- DeepDuh 14y agoWhy do you 'want to love' Heroku? Because their marketing speak is so great?
- DigitalJack 14y agoI'm sure it's because the idea of heroku is so great.
- DeepDuh 14y agoAs others here have pointed out in different discussions: Ideas aren't worth much by themselves. It's the subset that's implementable of which it's the subset that has gotten implemented that's worth anything. If we just love ideas we should better become philosophers. Sorry to be a bit harsh, but I find it a bit shocking how even in this field where we can basically play god and do whatever we want and what we think is best on increasingly powerful bit-cruncher-and-storer-machines, so many here seem to behave like a herd of sheep and just do what 'everyone else' does. Just sit down for a moment and think! What are my requirements right now? What could be a requirement in the near future? What technologies are there which can help me? Am I sure about these feature? Better read up on it first! How difficult is it to get it to behave in ways that are or can be important for me? Now list that stuff down. If it's puzzling sleep over it, forget it for a few days. Then suddenly, for example under a hot shower you get an idea - that requirement I had isn't really one, I can solve it differently! Come back, take the now fitting piece of the puzzle and do your job in 20% of the time that would have been needed if you would just have blindly followed some path. That's how it usually works for me. Be picky, be exact, but be lazy. Now about that routing dispatcher problem: Couldn't we solve that in one to two weeks on a generic plattform, but specifically for a certain use case? Let's say you want to have a worker queue of rails request handlers that work in parallel. Just write that damn router! Maybe I'd be lazy, learn Erlang for a week and think about it afterwards.
- ajanuary 14y agoThey want to love it because the idea is great. They don't love it because it doesn't deliver. Sounds like you both agree.