4 ms·
Good questions. Couple of things to note: 1. My paid features are all built on the core APIs that are open source. Almost every feature I sell has an OSS var
by mperham 10y ago
Good questions. Couple of things to note:
1. My paid features are all built on the core APIs that are open source. Almost every feature I sell has an OSS variant that someone has written. If you want to plug in a dozen different OSS libraries, integrate and test them together, you can get most of what I sell for $0. You will get varying levels of support and yearly upgrades of that stack will take days or weeks to validate. My customers recognize that building and maintaining any system like Sidekiq is non-trivial. You can spend a week or more annually maintaining your own Sidekiq "distro" or you can pay me to do it. So far, many hundreds of companies have opted for the latter.
2. Regarding multi-process specifically, each Sidekiq process is multi-threaded and will execute (by default) 25 jobs concurrently. Multi-threading is the OSS feature and delivers a huge jump in efficiently versus the old single-threaded Ruby job processors (Resque, DJ). multi-process makes it easier to spread work across many cores with a single command but nothing is stopping users from running multiple Sidekiq processes, they just won't share memory and must be managed separately.
- protomikron 10y agoOk, thanks for your answers, that sounds reasonable. So if I have a multi-core machine with 8 cores I can utilize all of them if I have 8 CPU intensive jobs (considering they fit into memory)?
- mperham 10y agoNot with MRI. The standard Ruby interpreter, like Python, uses a global lock to prevent threads from executing in parallel. You'd need to spin up 8 Sidekiq processes by hand or use the multi-process feature to automate that manual process. That said, your scenario is unusual. Most server-side jobs have plenty of I/O (think DB queries, memcached/redis calls, file I/O, etc), which allow the threads to context-switch; lots of I/O concurrency means lots of jobs get done quickly, even with one core.