6 ms·

You are completely wrong. GIL is released on I/O. So threads can parallelize during I/O calls. This means if your application is bottle necked by IO threading

You are completely wrong. GIL is released on I/O. So threads can parallelize during I/O calls. This means if your application is bottle necked by IO threading is better. If it's bottle necked by both IO and compute or only compute then multiple processes are better. For your specific case it sounds like the latter, your application is bound by both, therefore multiple processes are better, but what you are implying with your post of using multiprocessing over multi threading as the more "correct" way to "speed up" IO is categorically wrong. Dealing with the IO bottleneck by allowing IO calls to make progress while switching to other threads is a very common pattern in web development and the pattern has literally been baked into several frameworks and languages. NodeJS, GoLang, Erlang and Elixir can handle around 10k concurrent IO calls on a few threads without skipping a beat.

quietbritishjim 6y ago
> If it's bottle necked by both IO and compute or only compute then multiple processes are better. This is why I started this whole thread. The myth that you can't multithread compute in Python is so pervasive that even when we're in the middle of a thread specifically about the topic, and right next to my comment where I show hashlib specifically does release the GIL, there's still a comment saying that multiple processes are needed for compute parallelism.
- Geminidog 6y ago
  My comment is more addressing the guy trying to use processing OVER threading to improve overall speed of completing multiple IO tasks. Yeah I'm aware of your comment. And I can see how you could be pissed off over someone not mentioning what you're saying but please be aware that forum threads can go off into slight tangents when someone states something that is factually wrong. Yes. I know... Programmers can turn to C or they have to use specific libraries to achieve parallelism with threading. I get it. Though you're describing special cases. Similar to the grammatical special cases in English: https://www.e-education.psu.edu/styleforstudents/c1_p6.html https://www.e-education.psu.edu/styleforstudents/c1_p6.html Special cases make certain grammatical rules technically incorrect but the general notion behind those rules and why those rules are still pervasive and stated by people who are aware of the special cases is still valid. It's the same story with the python GIL. There could be multitudes of special cases but this does not negate the existence of a general grammatical rule: Compute in python threads is not parallel. Do I go into a long winded technical discussion of all the special cases or do I just use the generality to point out his categorical mistake? I guess I should have taken the long winded route because if I don't I'll piss you off. Thank you for voting me down btw, I will be sure to address all stakeholders in HN threads in my future replies.
jacobwilliamroy 6y ago
I think you're confusing async and parallelism. In multithreading, the computer will switch between multiple tasks whenever there is nothing to do, such as waiting for I/O read or waiting for a network packet to arrive. Things don't happen at the same time, it's just a more efficient, dynamic scheduling of a sequence of tasks happening one after the other. It's not physically possible for a multiple threads to happen at the same time unless they are on separate cores. In my use-case, hashing 200000 files on 7 separate hard drives, the only way to speed up the computation was to read from all 7 drives AT THE SAME TIME. There is no way to schedule 10^6 disk reads and hash operations into a single thread of execution that will reduce the runtime. Those async tricks can work in a more complicated program but you have to understand, I was ONLY reading from disk. The hashing step was faster than disk read (thanks xxhash) and the data was written to a sqlite file in a RAMdisk instantly so pretty much the entire runtime was spent reading from disk. multiprocessing was able to parallelize that unambiguously. Personally I don't trust python's multithreading because I have no control over whether it executes on one core or multiple cores, and in the world of parallel (not async) it is a very common convention that processes are parallel, while threads are async. However, that being said, I'm open-minded. I will try to benchmark reading from multiple hdds using multithreading and compare bandwidth to the multiprocessing approach. How's that?
- Geminidog 6y ago
  No you're confused. I'm perfectly aware of the differences between parallelism and concurrency. You are not. Have you ever heard of a popular web server framework called NodeJS? You can create servers with NodeJS that handle 10k concurrent IO requests. Let's say each request takes about 1s to finish because of the internet. If you fire 10k of these requests at a NodeJS server, the server can echo the ALL requests back in probably 2s. And get this. NodeJS is single threaded. If you fire 10k requests and NONE of the requests are parallel, you would typically expect all requests to finish at 10k seconds. This is not what happens with a single thread of NodeJS. People write entire chat servers in NodeJS that handle IO requests that hit around 10k concurrent messages in flight. NodeJS handles ALL of this on a SINGLE thread and from the users perspective all these requests finish in seconds. If you don't understand why the above happens it means you don't understand concurrency vs. parallelism in the context of IO. >that processes are parallel, while threads are async. No. Both processes and threads are async. Only processes are parallel on compute (for python) and threads become parallel on only IO (for python). And technically certain python libraries can parallelize threads. That is the key. A single thread can handle parallel IO. Why? Because the time spent on an IO operation is usually waiting for an inflight message to travel across the wire. So in a sense you can have 50 messages in flight across a wire handled by a single thread while the CPU just spends all this time waiting for the message to arrive. It's a form of "Parallelism" if you will but you won't find any literature using the word "parallelism" in conjunction with single threaded IO even though this is technically what is going on. >However, that being said, I'm open-minded. I will try to benchmark reading from multiple hdds using multithreading and compare bandwidth to the multiprocessing approach. How's that? This is a useless test. Your code involves both compute and IO. So processes will parallelize BOTH IO and compute. Threads will only parallelize IO, so if you have say like 10 fixed threads and 10 fixed processes OF COURSE processes will beat threads for your test case. To make it a fair test and to see the benefits of python threads over processes you need to get rid of the hash operation. Maybe make your python program write a copy the file to another file. Then try to make everything concurrent. By everything I mean for every single file, fire off a new thread and do the same for processes. You will find that the threaded approach will take up much less resources and go much further before crashing. Even better than threading though... high IO operations can actually be handled by a single thread. You can parallelize 200000 IO calls on a single thread with python async await or nodejs (i recommend node for this type of stuff). Though the other bottleneck will be your HD in this case. When your HDs see 200000 IO calls the HD's themselves will start serializing the requests. Also with such short message flight time in the wire is super short. Likely your IO is spending a good chunk of time writing messages to program memory as well. The most fair test is 200000 IO calls to multiple external services that can handle such volume. Don't touch your HD. Just focus tests on services that will not block or serialize IO. >multiprocessing was able to parallelize that unambiguously. No dude. Wrong again. Have you noticed that you're able to create more processes than you have cores? Have you ever thought about all the processes that are running on your OS? Likely more total processes then you have cores. How does this happen? Because not all processes are parallel. For OS threads and for all processes your OS decides whether they will be parallel or whether they will not. I know of no API that gives you direct control over which Core to execute your thread/process on. The only difference between threads and processes is that threads have shared memory with other threads and are less expensive to instantiate in terms of time and memory and processes have their own memory space and are therefore more "expensive". Python is unique in the sense that threads are not parallel on compute due to the GIL, but this is only true for python and python-like languages. How's that for a wall of text? Hope you learned something junior.

quietbritishjim 6y ago

> If it's bottle necked by both IO and compute or only compute then multiple processes are better. This is why I started this whole thread. The myth that you can't multithread compute in Python is so pervasive that even when we're in the middle of a thread specifically about the topic, and right next to my comment where I show hashlib specifically does release the GIL, there's still a comment saying that multiple processes are needed for compute parallelism.

Geminidog 6y ago
My comment is more addressing the guy trying to use processing OVER threading to improve overall speed of completing multiple IO tasks. Yeah I'm aware of your comment. And I can see how you could be pissed off over someone not mentioning what you're saying but please be aware that forum threads can go off into slight tangents when someone states something that is factually wrong. Yes. I know... Programmers can turn to C or they have to use specific libraries to achieve parallelism with threading. I get it. Though you're describing special cases. Similar to the grammatical special cases in English: https://www.e-education.psu.edu/styleforstudents/c1_p6.html https://www.e-education.psu.edu/styleforstudents/c1_p6.html Special cases make certain grammatical rules technically incorrect but the general notion behind those rules and why those rules are still pervasive and stated by people who are aware of the special cases is still valid. It's the same story with the python GIL. There could be multitudes of special cases but this does not negate the existence of a general grammatical rule: Compute in python threads is not parallel. Do I go into a long winded technical discussion of all the special cases or do I just use the generality to point out his categorical mistake? I guess I should have taken the long winded route because if I don't I'll piss you off. Thank you for voting me down btw, I will be sure to address all stakeholders in HN threads in my future replies.

jacobwilliamroy 6y ago

I think you're confusing async and parallelism. In multithreading, the computer will switch between multiple tasks whenever there is nothing to do, such as waiting for I/O read or waiting for a network packet to arrive. Things don't happen at the same time, it's just a more efficient, dynamic scheduling of a sequence of tasks happening one after the other. It's not physically possible for a multiple threads to happen at the same time unless they are on separate cores. In my use-case, hashing 200000 files on 7 separate hard drives, the only way to speed up the computation was to read from all 7 drives AT THE SAME TIME. There is no way to schedule 10^6 disk reads and hash operations into a single thread of execution that will reduce the runtime. Those async tricks can work in a more complicated program but you have to understand, I was ONLY reading from disk. The hashing step was faster than disk read (thanks xxhash) and the data was written to a sqlite file in a RAMdisk instantly so pretty much the entire runtime was spent reading from disk. multiprocessing was able to parallelize that unambiguously. Personally I don't trust python's multithreading because I have no control over whether it executes on one core or multiple cores, and in the world of parallel (not async) it is a very common convention that processes are parallel, while threads are async. However, that being said, I'm open-minded. I will try to benchmark reading from multiple hdds using multithreading and compare bandwidth to the multiprocessing approach. How's that?

Geminidog 6y ago
No you're confused. I'm perfectly aware of the differences between parallelism and concurrency. You are not. Have you ever heard of a popular web server framework called NodeJS? You can create servers with NodeJS that handle 10k concurrent IO requests. Let's say each request takes about 1s to finish because of the internet. If you fire 10k of these requests at a NodeJS server, the server can echo the ALL requests back in probably 2s. And get this. NodeJS is single threaded. If you fire 10k requests and NONE of the requests are parallel, you would typically expect all requests to finish at 10k seconds. This is not what happens with a single thread of NodeJS. People write entire chat servers in NodeJS that handle IO requests that hit around 10k concurrent messages in flight. NodeJS handles ALL of this on a SINGLE thread and from the users perspective all these requests finish in seconds. If you don't understand why the above happens it means you don't understand concurrency vs. parallelism in the context of IO. >that processes are parallel, while threads are async. No. Both processes and threads are async. Only processes are parallel on compute (for python) and threads become parallel on only IO (for python). And technically certain python libraries can parallelize threads. That is the key. A single thread can handle parallel IO. Why? Because the time spent on an IO operation is usually waiting for an inflight message to travel across the wire. So in a sense you can have 50 messages in flight across a wire handled by a single thread while the CPU just spends all this time waiting for the message to arrive. It's a form of "Parallelism" if you will but you won't find any literature using the word "parallelism" in conjunction with single threaded IO even though this is technically what is going on. >However, that being said, I'm open-minded. I will try to benchmark reading from multiple hdds using multithreading and compare bandwidth to the multiprocessing approach. How's that? This is a useless test. Your code involves both compute and IO. So processes will parallelize BOTH IO and compute. Threads will only parallelize IO, so if you have say like 10 fixed threads and 10 fixed processes OF COURSE processes will beat threads for your test case. To make it a fair test and to see the benefits of python threads over processes you need to get rid of the hash operation. Maybe make your python program write a copy the file to another file. Then try to make everything concurrent. By everything I mean for every single file, fire off a new thread and do the same for processes. You will find that the threaded approach will take up much less resources and go much further before crashing. Even better than threading though... high IO operations can actually be handled by a single thread. You can parallelize 200000 IO calls on a single thread with python async await or nodejs (i recommend node for this type of stuff). Though the other bottleneck will be your HD in this case. When your HDs see 200000 IO calls the HD's themselves will start serializing the requests. Also with such short message flight time in the wire is super short. Likely your IO is spending a good chunk of time writing messages to program memory as well. The most fair test is 200000 IO calls to multiple external services that can handle such volume. Don't touch your HD. Just focus tests on services that will not block or serialize IO. >multiprocessing was able to parallelize that unambiguously. No dude. Wrong again. Have you noticed that you're able to create more processes than you have cores? Have you ever thought about all the processes that are running on your OS? Likely more total processes then you have cores. How does this happen? Because not all processes are parallel. For OS threads and for all processes your OS decides whether they will be parallel or whether they will not. I know of no API that gives you direct control over which Core to execute your thread/process on. The only difference between threads and processes is that threads have shared memory with other threads and are less expensive to instantiate in terms of time and memory and processes have their own memory space and are therefore more "expensive". Python is unique in the sense that threads are not parallel on compute due to the GIL, but this is only true for python and python-like languages. How's that for a wall of text? Hope you learned something junior.