3 ms·
Using multiple processes in Python (rather than multiple threads) avoids many of the limitations of the GIL, and the new concurrent package (according to the PE
by mmavnn 16y ago
Using multiple processes in Python (rather than multiple threads) avoids many of the limitations of the GIL, and the new concurrent package (according to the PEP) can use the processing module or the threading module.
Processes in Python does have a different set of limitations in it's own right, but for most applications they aren't too disastrous.
- wladimir 16y agoThe shared-nothing approach (for example, implemented by using processes instead of threads) is also more scalable. After all, if you design your parallelism architecture to not rely on a shared memory space, you can later parallelize over multple machines as well as cores. The only thing I wonder about is that processes might be more heavy-weight than threads with respect to memory usage. Then again, if you only create a fixed amount of processes, equal to the number of cores, as you should for "perfect" parallelism, the overhead is pretty much zero.