3 ms·

How do you "approximate" stuff like retrying failed jobs and persisting them to disk then? You use whatever the Elixir community came up with which is Oban that

How do you "approximate" stuff like retrying failed jobs and persisting them to disk then? You use whatever the Elixir community came up with which is Oban that uses PG or Exq that uses redis. Sure hobby projects won't need an actual queue. You don't need Sidekiq for toy projects as well with Rails simply use the memory store, it can take you pretty far.

weatherlight 5y ago
literally on the Exq github readme: https://github.com/akira/exq#do-you-need-exq https://github.com/akira/exq#do-you-need-exq "While you may reach for Sidekiq / Resque / Celery by default when writing apps in other languages, in Elixir there are some good options to consider that are already provided by the language and platform. So before adding Exq or any Redis backed queueing library to your application, make sure to get familiar with OTP and see if that is enough for your needs. Redis backed queueing libraries do add additional infrastructure complexity and also overhead due to serialization / marshalling, so make sure to evaluate whether it is an actual need or not. Some OTP related documentation to look at: GenServer: http://elixir-lang.org/getting-started/mix-otp/genserver.html http://elixir-lang.org/getting-started/mix-otp/genserver.htm... Task: https://hexdocs.pm/elixir/Task.html https://hexdocs.pm/elixir/Task.html GenStage: https://hexdocs.pm/gen_stage/GenStage.html https://hexdocs.pm/gen_stage/GenStage.html Supervisor: http://elixir-lang.org/getting-started/mix-otp/supervisor-and-application.html http://elixir-lang.org/getting-started/mix-otp/supervisor-an... OTP: http://erlang.org/doc/ http://erlang.org/doc/ If you need a durable jobs, retries with exponential backoffs, dynamically scheduled jobs in the future - that are all able to survive application restarts, then an externally backed queueing library such as Exq could be a good fit. If you are starting a brand new project, I would also take a look at Faktory. It provides language independent queueing system, which means this logic doesn't have to be implemented across different languages and can use a thin client such as faktory_worker_ex."

weatherlight 5y ago

literally on the Exq github readme: https://github.com/akira/exq#do-you-need-exq https://github.com/akira/exq#do-you-need-exq "While you may reach for Sidekiq / Resque / Celery by default when writing apps in other languages, in Elixir there are some good options to consider that are already provided by the language and platform. So before adding Exq or any Redis backed queueing library to your application, make sure to get familiar with OTP and see if that is enough for your needs. Redis backed queueing libraries do add additional infrastructure complexity and also overhead due to serialization / marshalling, so make sure to evaluate whether it is an actual need or not. Some OTP related documentation to look at: GenServer: http://elixir-lang.org/getting-started/mix-otp/genserver.html http://elixir-lang.org/getting-started/mix-otp/genserver.htm... Task: https://hexdocs.pm/elixir/Task.html https://hexdocs.pm/elixir/Task.html GenStage: https://hexdocs.pm/gen_stage/GenStage.html https://hexdocs.pm/gen_stage/GenStage.html Supervisor: http://elixir-lang.org/getting-started/mix-otp/supervisor-and-application.html http://elixir-lang.org/getting-started/mix-otp/supervisor-an... OTP: http://erlang.org/doc/ http://erlang.org/doc/ If you need a durable jobs, retries with exponential backoffs, dynamically scheduled jobs in the future - that are all able to survive application restarts, then an externally backed queueing library such as Exq could be a good fit. If you are starting a brand new project, I would also take a look at Faktory. It provides language independent queueing system, which means this logic doesn't have to be implemented across different languages and can use a thin client such as faktory_worker_ex."