3 ms·

Every time someone brings up the GIL and performance, the answer is pretty much always "use a different language for the places you need especially high perform

Every time someone brings up the GIL and performance, the answer is pretty much always "use a different language for the places you need especially high performance". All this sturm and drang about Python not being performant enough is missing the point. It's not supposed to be the answer to special cases where maximum performance is required! It's a general-purpose language that is built around community and easy readability and elimination of boilerplate, and be good-enough for a vast majority of usecases. Just like any language, there are usecases where it is not appropriate. Clearly, it has succeeded given its significant reach and longevity.

Spivak 5y ago
Most high-level languages implement their performance critical functions at a lower level. However, languages that rely on a GIL (Python and Ruby) or isolated memory spaces (V8) have an additional hurdle that if you want a model of concurrency with many threads acting simultaneously on a single shared memory space you have to do additional work. For Python you either have to have your library pause execution of Python bytecode and do multithreaded things on your memory space, allocate an isolated memory arena for your multithreaded work, copy data in and out of it, and then spawn non-Python threads (see PEP 554 for an IRL example of this idea with the Python interpreter itself), or copy data to another process which can do the multithreaded things. With PEP 554 (right now using the C API) I personally have no issue with multithreaded Python since the overhead of copying memory between interpreters is fast enough for my work but it is overhead.
typon 5y ago
There is nothing about the Python language spec (as gleaned from looking at how CPython works) that forces it to be slow or not handle parallelism. In fact the addition of the massive amounts of library code and language changes to support async show that Python isn't even immune to preventing added complexity.

Spivak 5y ago

Most high-level languages implement their performance critical functions at a lower level. However, languages that rely on a GIL (Python and Ruby) or isolated memory spaces (V8) have an additional hurdle that if you want a model of concurrency with many threads acting simultaneously on a single shared memory space you have to do additional work. For Python you either have to have your library pause execution of Python bytecode and do multithreaded things on your memory space, allocate an isolated memory arena for your multithreaded work, copy data in and out of it, and then spawn non-Python threads (see PEP 554 for an IRL example of this idea with the Python interpreter itself), or copy data to another process which can do the multithreaded things. With PEP 554 (right now using the C API) I personally have no issue with multithreaded Python since the overhead of copying memory between interpreters is fast enough for my work but it is overhead.

typon 5y ago

There is nothing about the Python language spec (as gleaned from looking at how CPython works) that forces it to be slow or not handle parallelism. In fact the addition of the massive amounts of library code and language changes to support async show that Python isn't even immune to preventing added complexity.