4 ms·
Celery has a worker pool of separate Python processes that jobs can be offloaded to. It side steps the GIL because it doesn't use threading.
by darpa_escapee 10y ago
Celery has a worker pool of separate Python processes that jobs can be offloaded to. It side steps the GIL because it doesn't use threading.
- brianwawok 10y agoWell two ways. Either launch 32 or 64 copies of the process using multiprocessing, or 1000 threads with geventlet. I have never found a box I couldn't peg :) I guess if you had 1 gb ram and 16 cores it would be challenging in python.. but a few gb of ram and we are on.