3 ms·
Sure, with multiprocessing you can get 96 python processes running at 100% CPU while sharing a large dataset. Only problem is that 99% of that CPU usage is for
by apelapan 3y ago
Sure, with multiprocessing you can get 96 python processes running at 100% CPU while sharing a large dataset.
Only problem is that 99% of that CPU usage is for serializing/deserializing IPC messages and total throughput would have been higher using a single process.
There are use-cases for multiprocessing. As long as data sharing between processes is insignificant, it can be quite performant. Just like using a bash-wrapper script that orchestrates a bunch of python (or other) processes.