3 ms·
The standard Python workaround to the GIL is multiprocessing. Multiprocessing is basically a library which spins up a distributed system running locally - it fo
by yason 13y ago
The standard Python workaround to the GIL is multiprocessing. Multiprocessing is basically a library which spins up a distributed system running locally - it forks your process, and runs workers in the forks. The parent process then communicates with the child processes via unix pipes, TCP, or some such method, allowing multiple cores to be used.
Multiprocessing is the way to do parallelism. Deviating from that should be an exception -- for example, shared memory maps could be used to transfer select data objects instantaneously between the processes instead of serializing/deserializing over a pipe, and only those while still retaining separate process images. Threads were practically invented as a compensation for systems with heavy process image overhead.
I think Python is very Unix in this regard. And that's not a bad thing per se. Unix and Linux can do multiprocessing very efficiently.
- dakimov 13y agoOrly? If your language is handicapped maybe. In normal languages you have no problems with threads, and the assertion that multiprocessing is the way makes no sense.
- zbowling 13y ago> Multiprocessing is the way to do parallelism. Deviating from that should be an exception Unless you have a language that doesn't break down when you use threads. Threads are unquestionably easier and better to use for multiprocessing when they work without the tools you are using breaking that.
- yummyfajitas 13y agoMultiprocessing is the way to do parallelism. Nowadays I use Scala/Akka for concurrent event processing. As far as code organization goes it is a lot like multiprocessing. Just s/actor/process/. But unlike unix style forking/IPC, I don't have to waste memory/cpu cycles serializing/deserializing immutable messages just so another process can have a duplicated copy of it.