3 ms·

> Or when you have more and more complexity moved into async jobs How does this cause pain in rails specifically? Any 3 tier architecture is going to have the

> Or when you have more and more complexity moved into async jobs How does this cause pain in rails specifically? Any 3 tier architecture is going to have the same headaches.

berkes 5y ago
> How does this cause pain in rails specifically As long as the jobs are self-contained, cohesive and decoupled, there's no problem. They hardly ever are decoupled. Often, jobs will reach back into the model-layer (rather than receiving all data with which to work). You are now coupling yet another concern tightly to your Active-records. In practice, e.g. you have 400 jobs waiting, which will do a `deliver_to(Report.find(report_id).recipient.email)` What happens if your domain-model Report changes to have `recepients` instead? They hardly ever are cohesive. It's hard to orchestrate jobs - the tooling is simply not there in Rails-. So having tiny jobs spawning other jobs and/or sending eachother messages is not the common way: instead we build Giant Balls Of Sequential Code that perform a job. e.g. that ReportJob above will now (1) collect data, (2) prepare and filter it (3) push it into a template, (4) build a PDF from it, then (5) upload that to storage and (6) build an email and attach it. That is violating SRP on so many levels it hurts to write it down. And they are hardly ever self-contained. As mentioned above, many jobs will reach back into the models rather than having all the data in their job. Worse: jobs in Rails lack own state and management thereof. So they'll often push (intermediate) state through your models into your database. e.g. that ReportJob may trigger a statemachine changes with things like @report.transition_to(:pdf_generated).save!. Causing all sorts of issues with transactional boundaries (what if an HTTP request comes in that updates the same Reports' name, while you are building a PDF from it?) and thus race-conditions, locking and other issues. ActiveJob and the likes are very good as an aside on a CRUDdy app. To handle simple commands like notifications, emails, uploads, etc. But they lack severely when your domain is mostly command-based. In those cases, architectures such as Event-sourcing or Actors are a much better fit.
- nickjj 5y ago
  > And they are hardly ever self-contained. As mentioned above, many jobs will reach back into the models rather than having all the data in their job This is usually for a very good reason. If you passed in a user model into your job directly and then your job executed with that user there's a race condition here waiting to happen where the user might have been altered in between when the user clicked the button to issue the job and then job itself executing. This could have very bad side effects like the user receiving an email that they opt'd out of or maybe the user doesn't even exist in the system anymore or your job saves the user back to the DB with stale data that the user updated since then. It's super common and a best practice to send a unique ID of the user as a job argument and do the user lookup in the job at execution time of the job. This is typically something you'd do in any system, not just Rails. You mention "actors" which hints at Elixir but Oban (Elixir's most popular job processing library) also uses this approach of passing IDs because this problem isn't related to your tech stack. Also, you listed out 6 sequential steps in your ReportJob example. Based on what you wrote it sounds like they need to be done sequentially and can't be further parallelized. Based on no other context and it being a typical web app I'd keep that as 1 job and have 6 functions calls in the job that have the implementation details for each step so the job itself is very readable. I wouldn't make a jumbled mess of jobs-calling-jobs spaghetti there.

> Or when you have more and more complexity moved into async jobs How does this cause pain in rails specifically? Any 3 tier architecture is going to have the same headaches.

berkes 5y ago

> How does this cause pain in rails specifically As long as the jobs are self-contained, cohesive and decoupled, there's no problem. They hardly ever are decoupled. Often, jobs will reach back into the model-layer (rather than receiving all data with which to work). You are now coupling yet another concern tightly to your Active-records. In practice, e.g. you have 400 jobs waiting, which will do a `deliver_to(Report.find(report_id).recipient.email)` What happens if your domain-model Report changes to have `recepients` instead? They hardly ever are cohesive. It's hard to orchestrate jobs - the tooling is simply not there in Rails-. So having tiny jobs spawning other jobs and/or sending eachother messages is not the common way: instead we build Giant Balls Of Sequential Code that perform a job. e.g. that ReportJob above will now (1) collect data, (2) prepare and filter it (3) push it into a template, (4) build a PDF from it, then (5) upload that to storage and (6) build an email and attach it. That is violating SRP on so many levels it hurts to write it down. And they are hardly ever self-contained. As mentioned above, many jobs will reach back into the models rather than having all the data in their job. Worse: jobs in Rails lack own state and management thereof. So they'll often push (intermediate) state through your models into your database. e.g. that ReportJob may trigger a statemachine changes with things like @report.transition_to(:pdf_generated).save!. Causing all sorts of issues with transactional boundaries (what if an HTTP request comes in that updates the same Reports' name, while you are building a PDF from it?) and thus race-conditions, locking and other issues. ActiveJob and the likes are very good as an aside on a CRUDdy app. To handle simple commands like notifications, emails, uploads, etc. But they lack severely when your domain is mostly command-based. In those cases, architectures such as Event-sourcing or Actors are a much better fit.

nickjj 5y ago
> And they are hardly ever self-contained. As mentioned above, many jobs will reach back into the models rather than having all the data in their job This is usually for a very good reason. If you passed in a user model into your job directly and then your job executed with that user there's a race condition here waiting to happen where the user might have been altered in between when the user clicked the button to issue the job and then job itself executing. This could have very bad side effects like the user receiving an email that they opt'd out of or maybe the user doesn't even exist in the system anymore or your job saves the user back to the DB with stale data that the user updated since then. It's super common and a best practice to send a unique ID of the user as a job argument and do the user lookup in the job at execution time of the job. This is typically something you'd do in any system, not just Rails. You mention "actors" which hints at Elixir but Oban (Elixir's most popular job processing library) also uses this approach of passing IDs because this problem isn't related to your tech stack. Also, you listed out 6 sequential steps in your ReportJob example. Based on what you wrote it sounds like they need to be done sequentially and can't be further parallelized. Based on no other context and it being a typical web app I'd keep that as 1 job and have 6 functions calls in the job that have the implementation details for each step so the job itself is very readable. I wouldn't make a jumbled mess of jobs-calling-jobs spaghetti there.