3 ms·

Isn't multiprocessing almost syntactically equivalent to--but much more efficient than--threading in python?

• chrisseaton 9y ago
  Syntactically equivalent? The syntax is the same - both multiprocessing and threading are achieved through libraries and VM primitives, not by additional syntax. But if you meant semantically equivalent, no they aren't - threading is shared memory and multiprocessing is message passing. If you were to draw a tree of approaches to concurrency and parallelism, you would probably make this division the root node bisecting the domain into two - it's the two fundamental approaches to doing concurrency.
  • pvg 9y ago
    It's an implementation difference with big practical implications. 'Fundamental divide at the root of all concurrency' seems more than a little overstated.
    • chrisseaton 9y ago
      Almost every approach to concurrency fits into either shared memory or message passing. Those that cross between the two are usually notable for doing that and describe themselves as a hybrid approach. I think it divides the literature pretty cleanly in two.
  • vitus 9y ago
    > threading is shared memory and multiprocessing is message passing In most cases, yes. You can use shared memory buffers for IPC, though, although it typically requires more effort.
• wdroz 9y ago
  The problem is the GIL : https://wiki.python.org/moin/GlobalInterpreterLock https://wiki.python.org/moin/GlobalInterpreterLock Multiprocessing allows you to use all cores, but you can't share memory like in threads. So you have to Serialize/Deserialize a lot for exchange data. If your serializations cost are low and need computation power, you should use multiprocessing. If you are just waiting I/O you can use threads. BTW, modern python "ProcessPoolExecutor" is not designed for compute heavy batch like "Pool" from "multiprocessing".