4 ms·

You can of course use multithreading to get parallelism, eg. if the expensive calculations are not done in Python itself but rather some other library, be it cr

by dom0 10y ago

You can of course use multithreading to get parallelism, eg. if the expensive calculations are not done in Python itself but rather some other library, be it cryptography, compression, numpy, serdes'ing stuff and so on. I/O works in parallel as well.

ddorian43 10y ago
It's still 1 core for multithreading (on the python part).
- gshulegaard 10y ago
  Just to add more context for others... C extensions called from Python are still subject to the GIL of the calling interpreter[1]. In fact, this guaranteed thread safety has led to the creation of many non-thread safe Python C extensions which is now a critical blocker to efforts to removing the GIL[2], although there are also a number of other issues as well. As noted in the StackOverflow, you can release the GIL but there are some tricks to doing so safely[3]. But threads of a C extensions library are not necessarily pinned to a single core by the GIL like regular Python byte code. Although many performance oriented libraries like numpy[4] forsake multi-threading by default in many cases and rely on parallelization by the user (programmer)[5]. [1] http://stackoverflow.com/questions/651048/concurrency-are-python-extensions-written-in-c-c-affected-by-the-global-inter http://stackoverflow.com/questions/651048/concurrency-are-py... [2] https://youtu.be/P3AyI_u66Bw https://youtu.be/P3AyI_u66Bw [3] https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock https://docs.python.org/3/c-api/init.html#thread-state-and-t... [4] http://www.numpy.org/ http://www.numpy.org/ [5] http://stackoverflow.com/questions/16617973/why-isnt-numpy-mean-multithreaded http://stackoverflow.com/questions/16617973/why-isnt-numpy-m...
jMyles 10y ago
I wish I had a thousand upvotes to give this comment. This is the single most misunderstood concept in async python. I started and still contribute to hendrix (a "Twisted Django") and I can't believe how bright a lightbulb this is for people when they realize it.

by dom0 10y ago

ddorian43 10y ago

It's still 1 core for multithreading (on the python part).

gshulegaard 10y ago
Just to add more context for others... C extensions called from Python are still subject to the GIL of the calling interpreter[1]. In fact, this guaranteed thread safety has led to the creation of many non-thread safe Python C extensions which is now a critical blocker to efforts to removing the GIL[2], although there are also a number of other issues as well. As noted in the StackOverflow, you can release the GIL but there are some tricks to doing so safely[3]. But threads of a C extensions library are not necessarily pinned to a single core by the GIL like regular Python byte code. Although many performance oriented libraries like numpy[4] forsake multi-threading by default in many cases and rely on parallelization by the user (programmer)[5]. [1] http://stackoverflow.com/questions/651048/concurrency-are-python-extensions-written-in-c-c-affected-by-the-global-inter http://stackoverflow.com/questions/651048/concurrency-are-py... [2] https://youtu.be/P3AyI_u66Bw https://youtu.be/P3AyI_u66Bw [3] https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock https://docs.python.org/3/c-api/init.html#thread-state-and-t... [4] http://www.numpy.org/ http://www.numpy.org/ [5] http://stackoverflow.com/questions/16617973/why-isnt-numpy-mean-multithreaded http://stackoverflow.com/questions/16617973/why-isnt-numpy-m...

jMyles 10y ago

I wish I had a thousand upvotes to give this comment. This is the single most misunderstood concept in async python. I started and still contribute to hendrix (a "Twisted Django") and I can't believe how bright a lightbulb this is for people when they realize it.