4 ms·

I'm not sure how that would solve the problem. Not every server is a front end web server. Think of something like video transcoding, or complex database queri

I'm not sure how that would solve the problem. Not every server is a front end web server. Think of something like video transcoding, or complex database queries. At some point you are going to have a number of servers that do the work, and a load balancer needs to balance the work between them. If one server is busy with a job that will take an hour, there's no point in randomly queueing up lots of work while others are idle.

rollcat 9y ago
Read up on how task queues work - they do fill a role of a load balancer, your video transcoding case is a perfect example application. Basically workers can leave/join the pool at any time, poll for more work whenever idle/done, and signal failures so that jobs can be retried later. There's no such thing as an idle worker, unless the queue is empty. No "overloaded" workers either, running two CPU-bound tasks concurrently won't get either of them done sooner. Your monitoring / auto-scaling system can watch the length of the task queue and add/remove workers as needed. (You can also try to combine this with AWS spot instances.)
- jakobegger 9y ago
  Okay, so that would require totally changing the application architecture, and in the end it would suffer from exactly the same issue: A broken worker that takes tasks from the queue and immediately returns an incorrect result will quickly take over a significant portion of the tasks. That’s exactly the issue the article was talking about.
  - rollcat 9y ago
    > A broken worker that takes tasks from the queue and immediately returns an incorrect result will quickly take over a significant portion of the tasks. Except now you can use a round-robin/"dumb" LB on the frontend, fixing that exact issue (and a whole class of other problems). Yes, a broken worker will quickly drain the queue, except once you find out and remove it, you can (hopefully) resubmit the failed jobs. It gives you a new primitive to work with - HTTP requests are 1:1 to HTTP responses, but a job's result can be updated, and the new result propagated up the chain. (Think: media transcoding / post-processing, rendering pipelines, data analysis, etc.) > that would require totally changing the application architecture True, usual cost/benefit assessment when dealing with technical debt, etc.

rollcat 9y ago

Read up on how task queues work - they do fill a role of a load balancer, your video transcoding case is a perfect example application. Basically workers can leave/join the pool at any time, poll for more work whenever idle/done, and signal failures so that jobs can be retried later. There's no such thing as an idle worker, unless the queue is empty. No "overloaded" workers either, running two CPU-bound tasks concurrently won't get either of them done sooner. Your monitoring / auto-scaling system can watch the length of the task queue and add/remove workers as needed. (You can also try to combine this with AWS spot instances.)

jakobegger 9y ago
Okay, so that would require totally changing the application architecture, and in the end it would suffer from exactly the same issue: A broken worker that takes tasks from the queue and immediately returns an incorrect result will quickly take over a significant portion of the tasks. That’s exactly the issue the article was talking about.
- rollcat 9y ago
  > A broken worker that takes tasks from the queue and immediately returns an incorrect result will quickly take over a significant portion of the tasks. Except now you can use a round-robin/"dumb" LB on the frontend, fixing that exact issue (and a whole class of other problems). Yes, a broken worker will quickly drain the queue, except once you find out and remove it, you can (hopefully) resubmit the failed jobs. It gives you a new primitive to work with - HTTP requests are 1:1 to HTTP responses, but a job's result can be updated, and the new result propagated up the chain. (Think: media transcoding / post-processing, rendering pipelines, data analysis, etc.) > that would require totally changing the application architecture True, usual cost/benefit assessment when dealing with technical debt, etc.