5 ms·

One aspect of this set up I’ve never been able to understand is how the application then gets the result from the worker? If it’s polling for status from the ba

One aspect of this set up I’ve never been able to understand is how the application then gets the result from the worker? If it’s polling for status from the backend doesn’t that defeat the purpose of having a worker to begin with? Or does this set up only work for tasks that don’t need to have the backend notified about the result so the front end can just poll for the result via the application?

leowoo91 6y ago
In case of a polling requirement from user perspective (e.g. waiting for a notification) it's true it doesn't matter. However, it allows application to respond to 'other' users. Whole point is to serve multiple users at the same time.
- procinct 6y ago
  What I mean is that if your backend is polling for a task to finish, that is also taking away time from other users. For some tasks that won’t matter because the backend won’t need to be aware of the outcome of the task but there could be some longer jobs that the backend needs to be aware of the outcome right away. You could get it so the polling is done on the front end and then passes the outcome to the backend but that obviously isn’t a good idea because then the backend is trusting outcome data from the front end.
  - leowoo91 6y ago
    I see your concern is focused on polling case (e.g a chat room). As goes with the McDonalds example, a single cashier can reply the question "is my order ready?" rapidly. While it takes seconds in real world, a client polling request would take milliseconds to complete, so it can still serve hundreds of clients in a given second. If you'd need more for that part, there can be more cashiers/apps to make "asking" part scale indefinitely (even you'd have 1 worker only).
WJW 6y ago
If you really need the answer immediately to show to a user on a page, background workers (usually) don't help. However, there are a bunch of situations where there is no feedback to the user other than "we received your request in good order and will see that it's done". For example, sending a mail can take a while because mailservers have queues and whatnot. If you send a mail after signing up a user for them to verify their email, you don't have to wait until the mail is "really" sent before letting them know that their signup has been processed. You can tell a background worker to send the mail and return to the user much faster. For another example, in a previous job I worked for a big file sharing service. If user wanted their files deleted that caused all sorts of calls to AWS to actually delete the files, which could take a while. However, from the user perspective it was pretty fast because all we had to do was set the file in the database to "delete in progress" state and tell a background worker to delete the files. Then we could show the user that their files were being deleted within a couple dozen milliseconds instead of having to wait for all the AWS calls to complete.
- 1337shadow 6y ago
  Not to mention the case where the mailserver is down or denies service, which will also happen at some point even if you have HA mailserver: be it with AWS emails, mailjet and whatnot. One day it'll fail everyone. Then, what's it going to be ? Return an HTTP 500 to the user rolling back the transaction ? Spooling emails removes that failure spot at all.
masklinn 6y ago
I would agree, generally a task queue makes sense for jobs which are not needed to respond to a query (e.g. generating reports or sending emails or whatever), otherwise you're just adding delays in the response chain. Although this delay chain might be considered worth it you don't want to scale the frontend to multiple workers for some reason e.g. single-threaded evented runtime, or GIL runtime (python, ocaml), or if you want to avoid CPU-hard tasks being executed on your frontends. In that case, it might be valuable to transform CPU waits into IO waits by moving the CPU work to a jobs queue, possibly running its workers on a different set of machines entirely.
adamcharnock 6y ago
You’re spot on really. Having the front end wait for a background task to complete broadly defeats the purpose. There are some caveats though: if you’re using an async/threaded web server then it may not matter that you have a pending request hanging around as your web server is free to continue serving other requests. It also may be that you need to run the task on specialist hardware for some reason. Really though, I think a lot of people use celery for offloading things like email sending and API calls which, IMHO, isn’t really worth the complexity (especially as SMTP is basically a queue anyway). Of course, YMMV depending on your use case. However, I find it is often more worthwhile for: 1. Tasks which take a long time to run 2. Tasks which need to happen on a schedule, rather than in response to a user request. There is an option 3 too, which is for inter-software communication. Eg events or RPCs, but I found Celery to be very much a square-peg-round-hole for this, which is why I developed Lightbus (http://lightbus.org http://lightbus.org). Lightbus also supports background tasks and scheduled tasks. /plug
- doliveira 6y ago
  It's not just about the time the operation takes, it's about reliability. Even if sending an email synchronously doesn't usually take more than a few milliseconds, you still need to handle cases like servers failing in the middle of the request, temporary upstream unavailability, some expired API, account limits reached, etc... Honestly, I think that mostly anything that doesn't depend directly in the current state of your infrastructure should be done asynchronously. I've had a lot of issues with systems that start up doing everything synchronously: you'll probably need to refactor it to be asynchronous in emergency mode during a crisis.
  - marcosdumay 6y ago
    For email, that's why you set a relay within your control, that will accept messages without a fuss and send them around following SMTP conventions. But anyway, how is your application supposed to respond after any of those failures? Is it just supposed to ignore the failure and thread on like if nothing happened, leaving your users on the dark? Is it supposed to reliably log every task so that it can retry anything that fails and in the worst case feed failures into some monitoring system/process? Or is it supposed to inform the user of any success or failure before the user can move on? Queue software is only a good match for the first. For the second you will need to roll your own interface with the monitoring system anyway, so it's much easier to roll your own queues and get control of everything. The third one is best done synchronous, it doesn't matter the nature of the process or how long it takes. But funny thing is, I have never seen the first situation on the wild.
golergka 6y ago
Right now, I'm designing a service that's very similar to OP, with workers waiting for an external API (or APIs) to answer, which can be slow sometimes. Looks like we're going to use websockets or polling to update the client with all the delivered data, and introduce yet another service for it, which would actually play the role of "LED screen" and ready order station in this analogy. But as we decided to implement an MVP with a single HTTP request from the client, this whole separation doesn't make any sense, exactly as you noted.
notyourday 6y ago
In my experience a lot of applications with workers that require a result to be furnished to a client are incorrectly engineered. The correct pattern should be a client submits a request to get something done to a thin layer, receives a ticket that allows it to claim the result and goes away to either check for the result via polling for the ticket or receives a call back.
doliveira 6y ago
Checking the result is just doing a SELECT in your database.
- procinct 6y ago
  I was more referring to how it knows once it’s finished since the task is asynchronous. So for the backend to find out, it could do the select you mentioned and find out it’s still in progress. Then it checks again next second and still in progress. Then checks again and it’s done. But now your backend has been polling for an update and so you might as well have performed the task in the application because it is still being used up for the duration of the task.
  - turtlebits 6y ago
    Have your worker POST back to your web server when it’s done. Use websockets to notify the user.
jb3689 6y ago
> If it’s polling for status from the backend doesn’t that defeat the purpose of having a worker to begin with? It depends. If you needed a coordination layer or you needed to isolate certain types of traffic then it makes more sense. I assume your alternative here is "why not just have a new client tier doing work" which is a reasonable architecture too > One aspect of this set up I’ve never been able to understand is how the application then gets the result from the worker? Often they don't in these architectures. I found it a little strange that a job queue was being used to serve (what seems like) synchronous traffic. Usually I see job queues in the wild used for async/send-and-forget workloads Personally I would rather chain multiple synchronous service calls if I needed a synchronous workflow. It's just simpler to me to stick web workers behind a load balancer and scale that. This is less elegant when things take a very long time though or are prone to retries. Either the client or server needs to be responsible for queuing/message persistence/retries - with services the client does it, with job queues the server does it

leowoo91 6y ago

In case of a polling requirement from user perspective (e.g. waiting for a notification) it's true it doesn't matter. However, it allows application to respond to 'other' users. Whole point is to serve multiple users at the same time.

procinct 6y ago
What I mean is that if your backend is polling for a task to finish, that is also taking away time from other users. For some tasks that won’t matter because the backend won’t need to be aware of the outcome of the task but there could be some longer jobs that the backend needs to be aware of the outcome right away. You could get it so the polling is done on the front end and then passes the outcome to the backend but that obviously isn’t a good idea because then the backend is trusting outcome data from the front end.
- leowoo91 6y ago
  I see your concern is focused on polling case (e.g a chat room). As goes with the McDonalds example, a single cashier can reply the question "is my order ready?" rapidly. While it takes seconds in real world, a client polling request would take milliseconds to complete, so it can still serve hundreds of clients in a given second. If you'd need more for that part, there can be more cashiers/apps to make "asking" part scale indefinitely (even you'd have 1 worker only).

WJW 6y ago

If you really need the answer immediately to show to a user on a page, background workers (usually) don't help. However, there are a bunch of situations where there is no feedback to the user other than "we received your request in good order and will see that it's done". For example, sending a mail can take a while because mailservers have queues and whatnot. If you send a mail after signing up a user for them to verify their email, you don't have to wait until the mail is "really" sent before letting them know that their signup has been processed. You can tell a background worker to send the mail and return to the user much faster. For another example, in a previous job I worked for a big file sharing service. If user wanted their files deleted that caused all sorts of calls to AWS to actually delete the files, which could take a while. However, from the user perspective it was pretty fast because all we had to do was set the file in the database to "delete in progress" state and tell a background worker to delete the files. Then we could show the user that their files were being deleted within a couple dozen milliseconds instead of having to wait for all the AWS calls to complete.

1337shadow 6y ago
Not to mention the case where the mailserver is down or denies service, which will also happen at some point even if you have HA mailserver: be it with AWS emails, mailjet and whatnot. One day it'll fail everyone. Then, what's it going to be ? Return an HTTP 500 to the user rolling back the transaction ? Spooling emails removes that failure spot at all.

masklinn 6y ago

I would agree, generally a task queue makes sense for jobs which are not needed to respond to a query (e.g. generating reports or sending emails or whatever), otherwise you're just adding delays in the response chain. Although this delay chain might be considered worth it you don't want to scale the frontend to multiple workers for some reason e.g. single-threaded evented runtime, or GIL runtime (python, ocaml), or if you want to avoid CPU-hard tasks being executed on your frontends. In that case, it might be valuable to transform CPU waits into IO waits by moving the CPU work to a jobs queue, possibly running its workers on a different set of machines entirely.

adamcharnock 6y ago

You’re spot on really. Having the front end wait for a background task to complete broadly defeats the purpose. There are some caveats though: if you’re using an async/threaded web server then it may not matter that you have a pending request hanging around as your web server is free to continue serving other requests. It also may be that you need to run the task on specialist hardware for some reason. Really though, I think a lot of people use celery for offloading things like email sending and API calls which, IMHO, isn’t really worth the complexity (especially as SMTP is basically a queue anyway). Of course, YMMV depending on your use case. However, I find it is often more worthwhile for: 1. Tasks which take a long time to run 2. Tasks which need to happen on a schedule, rather than in response to a user request. There is an option 3 too, which is for inter-software communication. Eg events or RPCs, but I found Celery to be very much a square-peg-round-hole for this, which is why I developed Lightbus (http://lightbus.org http://lightbus.org). Lightbus also supports background tasks and scheduled tasks. /plug

doliveira 6y ago
It's not just about the time the operation takes, it's about reliability. Even if sending an email synchronously doesn't usually take more than a few milliseconds, you still need to handle cases like servers failing in the middle of the request, temporary upstream unavailability, some expired API, account limits reached, etc... Honestly, I think that mostly anything that doesn't depend directly in the current state of your infrastructure should be done asynchronously. I've had a lot of issues with systems that start up doing everything synchronously: you'll probably need to refactor it to be asynchronous in emergency mode during a crisis.
- marcosdumay 6y ago
  For email, that's why you set a relay within your control, that will accept messages without a fuss and send them around following SMTP conventions. But anyway, how is your application supposed to respond after any of those failures? Is it just supposed to ignore the failure and thread on like if nothing happened, leaving your users on the dark? Is it supposed to reliably log every task so that it can retry anything that fails and in the worst case feed failures into some monitoring system/process? Or is it supposed to inform the user of any success or failure before the user can move on? Queue software is only a good match for the first. For the second you will need to roll your own interface with the monitoring system anyway, so it's much easier to roll your own queues and get control of everything. The third one is best done synchronous, it doesn't matter the nature of the process or how long it takes. But funny thing is, I have never seen the first situation on the wild.

golergka 6y ago

Right now, I'm designing a service that's very similar to OP, with workers waiting for an external API (or APIs) to answer, which can be slow sometimes. Looks like we're going to use websockets or polling to update the client with all the delivered data, and introduce yet another service for it, which would actually play the role of "LED screen" and ready order station in this analogy. But as we decided to implement an MVP with a single HTTP request from the client, this whole separation doesn't make any sense, exactly as you noted.

notyourday 6y ago

In my experience a lot of applications with workers that require a result to be furnished to a client are incorrectly engineered. The correct pattern should be a client submits a request to get something done to a thin layer, receives a ticket that allows it to claim the result and goes away to either check for the result via polling for the ticket or receives a call back.

doliveira 6y ago

Checking the result is just doing a SELECT in your database.

procinct 6y ago
I was more referring to how it knows once it’s finished since the task is asynchronous. So for the backend to find out, it could do the select you mentioned and find out it’s still in progress. Then it checks again next second and still in progress. Then checks again and it’s done. But now your backend has been polling for an update and so you might as well have performed the task in the application because it is still being used up for the duration of the task.
- turtlebits 6y ago
  Have your worker POST back to your web server when it’s done. Use websockets to notify the user.

jb3689 6y ago

> If it’s polling for status from the backend doesn’t that defeat the purpose of having a worker to begin with? It depends. If you needed a coordination layer or you needed to isolate certain types of traffic then it makes more sense. I assume your alternative here is "why not just have a new client tier doing work" which is a reasonable architecture too > One aspect of this set up I’ve never been able to understand is how the application then gets the result from the worker? Often they don't in these architectures. I found it a little strange that a job queue was being used to serve (what seems like) synchronous traffic. Usually I see job queues in the wild used for async/send-and-forget workloads Personally I would rather chain multiple synchronous service calls if I needed a synchronous workflow. It's just simpler to me to stick web workers behind a load balancer and scale that. This is less elegant when things take a very long time though or are prone to retries. Either the client or server needs to be responsible for queuing/message persistence/retries - with services the client does it, with job queues the server does it