4 ms·
The usual answer is: in the case of blocking I/O, the thread running send/recv can block while other python code runs. In practice, this doesn’t work particula
by boulos 7y ago
The usual answer is: in the case of blocking I/O, the thread running send/recv can block while other python code runs.
In practice, this doesn’t work particularly well, as you rarely have massively I/O bound things in Python.
- keypusher 7y agoNot sure what kind of applications you work on, but most web apps are IO bound (waiting on network calls).
- preordained 7y agoI'm not the person you were originally responding to, but I know Python is popular in data science and such where there would be a lot more tied up in pure computation/number crunching.
- thelastbender12 7y agoTo that point, most third-party libraries used also delegate the number crunching to C/C++ code, so most time is still spent outside the GIL.
- dodobirdlord 7y agoI know that you already got an answer to this effect, but to provide some more information, most data science workflows in Python rely heavily on calls into numerical libraries (numpy, scipy, pandas, tensorflow, pytorch, matplotlib) that are Python wrappers over compiled binaries (mostly C and Fortran, a not-inconsiderable amount of handwritten Assembly), that have been constructed so that the wrapper safely yields the GIL before invoking the underlying binary. This is all the more important when considering libraries like tensorflow or pytorch that may involve complex long-running interaction with training resources across a network. Control is yielded to allow the interpreter to continue carrying out tasks like displaying the ongoing training progress, or loading training data.
- ptx 7y agoThe program doesn't have to be massively I/O bound for threads to be useful. Even with only a UI thread and a single background thread they're useful for animating a progress indicator in the UI thread while waiting for a slow network request to finish in the background thread, which works just fine in Python.
- zbentley 7y ago> you rarely have massively I/O bound things in Python. Citation needed.
- boulos 7y agoI should have clarified: when I meant "doesn't work particularly well", I meant that its incredibly difficult to saturate say a 100 thread box. (Which with Rome isn't crazy!) Only being able to run a single thread of "logic" basically makes that impossible, as you need to usually do some computation to figure out what bytes to send/receive, do something with the results, and so on.