4 ms·
Multiprocessing requires multiple python processes which are expensive with regards to startup time and resource use. Multiple interpreters give you parallelis
by O5vYtytb 5y ago
Multiprocessing requires multiple python processes which are expensive with regards to startup time and resource use. Multiple interpreters give you parallelism without so much overhead.
- wruza 5y agoWait, isn’t python startup slow because of the interpreter startup and not process creation itself? A thread also has to be created (saves dll mapping time but that’s it) and python state also has to be initialized in a complete isolation on that thread (means setting up the entire scene again and loading stdlib from sources). Am I wrong?
- O5vYtytb 5y agoIt's a number of factors, including the loading of the modules necessary to start python and the standard library. Check out https://github.com/faster-cpython/ideas/issues?q=is%3Aissue+is%3Aopen+faster+startup https://github.com/faster-cpython/ideas/issues?q=is%3Aissue+... for some ideas regarding improvement there.
- maxnoe 5y agoThe point is each process has to start its own interpreter.
- wruza 5y agoThreads don’t magically start with a python interpreter that is ready to use, it has to be initialized for every thread out there separately as well, unless some clever immutability tricks are involved to skip big parts of that initialization.