3 ms·

I think you're confusing async and parallelism. In multithreading, the computer will switch between multiple tasks whenever there is nothing to do, such as wait

I think you're confusing async and parallelism. In multithreading, the computer will switch between multiple tasks whenever there is nothing to do, such as waiting for I/O read or waiting for a network packet to arrive. Things don't happen at the same time, it's just a more efficient, dynamic scheduling of a sequence of tasks happening one after the other. It's not physically possible for a multiple threads to happen at the same time unless they are on separate cores. In my use-case, hashing 200000 files on 7 separate hard drives, the only way to speed up the computation was to read from all 7 drives AT THE SAME TIME. There is no way to schedule 10^6 disk reads and hash operations into a single thread of execution that will reduce the runtime. Those async tricks can work in a more complicated program but you have to understand, I was ONLY reading from disk. The hashing step was faster than disk read (thanks xxhash) and the data was written to a sqlite file in a RAMdisk instantly so pretty much the entire runtime was spent reading from disk. multiprocessing was able to parallelize that unambiguously. Personally I don't trust python's multithreading because I have no control over whether it executes on one core or multiple cores, and in the world of parallel (not async) it is a very common convention that processes are parallel, while threads are async. However, that being said, I'm open-minded. I will try to benchmark reading from multiple hdds using multithreading and compare bandwidth to the multiprocessing approach. How's that?

Geminidog 6y ago
No you're confused. I'm perfectly aware of the differences between parallelism and concurrency. You are not. Have you ever heard of a popular web server framework called NodeJS? You can create servers with NodeJS that handle 10k concurrent IO requests. Let's say each request takes about 1s to finish because of the internet. If you fire 10k of these requests at a NodeJS server, the server can echo the ALL requests back in probably 2s. And get this. NodeJS is single threaded. If you fire 10k requests and NONE of the requests are parallel, you would typically expect all requests to finish at 10k seconds. This is not what happens with a single thread of NodeJS. People write entire chat servers in NodeJS that handle IO requests that hit around 10k concurrent messages in flight. NodeJS handles ALL of this on a SINGLE thread and from the users perspective all these requests finish in seconds. If you don't understand why the above happens it means you don't understand concurrency vs. parallelism in the context of IO. >that processes are parallel, while threads are async. No. Both processes and threads are async. Only processes are parallel on compute (for python) and threads become parallel on only IO (for python). And technically certain python libraries can parallelize threads. That is the key. A single thread can handle parallel IO. Why? Because the time spent on an IO operation is usually waiting for an inflight message to travel across the wire. So in a sense you can have 50 messages in flight across a wire handled by a single thread while the CPU just spends all this time waiting for the message to arrive. It's a form of "Parallelism" if you will but you won't find any literature using the word "parallelism" in conjunction with single threaded IO even though this is technically what is going on. >However, that being said, I'm open-minded. I will try to benchmark reading from multiple hdds using multithreading and compare bandwidth to the multiprocessing approach. How's that? This is a useless test. Your code involves both compute and IO. So processes will parallelize BOTH IO and compute. Threads will only parallelize IO, so if you have say like 10 fixed threads and 10 fixed processes OF COURSE processes will beat threads for your test case. To make it a fair test and to see the benefits of python threads over processes you need to get rid of the hash operation. Maybe make your python program write a copy the file to another file. Then try to make everything concurrent. By everything I mean for every single file, fire off a new thread and do the same for processes. You will find that the threaded approach will take up much less resources and go much further before crashing. Even better than threading though... high IO operations can actually be handled by a single thread. You can parallelize 200000 IO calls on a single thread with python async await or nodejs (i recommend node for this type of stuff). Though the other bottleneck will be your HD in this case. When your HDs see 200000 IO calls the HD's themselves will start serializing the requests. Also with such short message flight time in the wire is super short. Likely your IO is spending a good chunk of time writing messages to program memory as well. The most fair test is 200000 IO calls to multiple external services that can handle such volume. Don't touch your HD. Just focus tests on services that will not block or serialize IO. >multiprocessing was able to parallelize that unambiguously. No dude. Wrong again. Have you noticed that you're able to create more processes than you have cores? Have you ever thought about all the processes that are running on your OS? Likely more total processes then you have cores. How does this happen? Because not all processes are parallel. For OS threads and for all processes your OS decides whether they will be parallel or whether they will not. I know of no API that gives you direct control over which Core to execute your thread/process on. The only difference between threads and processes is that threads have shared memory with other threads and are less expensive to instantiate in terms of time and memory and processes have their own memory space and are therefore more "expensive". Python is unique in the sense that threads are not parallel on compute due to the GIL, but this is only true for python and python-like languages. How's that for a wall of text? Hope you learned something junior.
- jacobwilliamroy 6y ago
  I did the benchmark and multiprocessing achieved higher bandwidth than multithreading. 8 times higher. This was just reading from disk, so no hashing step at all. Didn't even write to disk. I think you might not be so knowledgable about concurrency.
  - Geminidog 6y ago
    Did you read my full post? I don't think you did. Let me quote it: >To make it a fair test and to see the benefits of python threads over processes you need to get rid of the hash operation. Maybe make your python program write a copy the file to another file. Then try to make everything concurrent. By everything I mean for every single file, fire off a new thread and do the same for processes. >Though the other bottleneck will be your HD in this case. When your HDs see 200000 IO calls the HD's themselves will start serializing the requests. Also with such short message flight time in the wire is super short. Likely your IO is spending a good chunk of time writing messages to program memory as well. >The most fair test is 200000 IO calls to multiple external services that can handle such volume. Don't touch your HD. Just focus tests on services that will not block or serialize IO. I'm very very knowledgeable about concurrency. The problem is you. You failed to read my post and the caveats of all the details I mentioned. You failed to adjust the test fully to make it fair. You just overall failed and you don't know much. Let me explain why you failed. Like I quoted above. HD IO has very very short time of flight meaning that the actual operation of compute (writing the data to program memory) takes much more time. Your threads aren't parallelizing that. Like I said before THE ONLY DIFFERENCE in processes and threads is that python threads DO NOT parallelize compute and processes are MORE EXPENSIVE to spawn in terms of memory space. SO if you logically parallelize off of a fix number of threads/processes what do you think will occur? Of course processes will be faster. I told you to adjust the test to spawn a new thread/process for every IO call to see the benefits. Because threads are cheaper in memory to spawn it will be able to spawn MANY more threads than it can processes. So if your operation is IO bound what you typically did with like 10 processes can likely be done with 1000 threads. <--- That is the difference. You said you have 2000000 files? That means 2000000 threads/processes must be spawned for a fair test. And if you can, try to hit some server on the internet that can handle that many requests to fully see parallelization of IO because time of flight is longer over the internet. Additionally your HDD's themselves can't parallelize 2000000 requests so in the end all your threads/processes will be bottlenecked by the HD itself NOT by IO. That's why I told you to hit something over the internet that can handle 2000000 IO calls. I mentioned All of this in my original response and you failed to understand. Now do you Get it? I actually am trying to teach you something and you have the balls to tell me that I don't know about concurrency. It's quite obvious you have no idea what you're talking about.

Geminidog 6y ago

No you're confused. I'm perfectly aware of the differences between parallelism and concurrency. You are not. Have you ever heard of a popular web server framework called NodeJS? You can create servers with NodeJS that handle 10k concurrent IO requests. Let's say each request takes about 1s to finish because of the internet. If you fire 10k of these requests at a NodeJS server, the server can echo the ALL requests back in probably 2s. And get this. NodeJS is single threaded. If you fire 10k requests and NONE of the requests are parallel, you would typically expect all requests to finish at 10k seconds. This is not what happens with a single thread of NodeJS. People write entire chat servers in NodeJS that handle IO requests that hit around 10k concurrent messages in flight. NodeJS handles ALL of this on a SINGLE thread and from the users perspective all these requests finish in seconds. If you don't understand why the above happens it means you don't understand concurrency vs. parallelism in the context of IO. >that processes are parallel, while threads are async. No. Both processes and threads are async. Only processes are parallel on compute (for python) and threads become parallel on only IO (for python). And technically certain python libraries can parallelize threads. That is the key. A single thread can handle parallel IO. Why? Because the time spent on an IO operation is usually waiting for an inflight message to travel across the wire. So in a sense you can have 50 messages in flight across a wire handled by a single thread while the CPU just spends all this time waiting for the message to arrive. It's a form of "Parallelism" if you will but you won't find any literature using the word "parallelism" in conjunction with single threaded IO even though this is technically what is going on. >However, that being said, I'm open-minded. I will try to benchmark reading from multiple hdds using multithreading and compare bandwidth to the multiprocessing approach. How's that? This is a useless test. Your code involves both compute and IO. So processes will parallelize BOTH IO and compute. Threads will only parallelize IO, so if you have say like 10 fixed threads and 10 fixed processes OF COURSE processes will beat threads for your test case. To make it a fair test and to see the benefits of python threads over processes you need to get rid of the hash operation. Maybe make your python program write a copy the file to another file. Then try to make everything concurrent. By everything I mean for every single file, fire off a new thread and do the same for processes. You will find that the threaded approach will take up much less resources and go much further before crashing. Even better than threading though... high IO operations can actually be handled by a single thread. You can parallelize 200000 IO calls on a single thread with python async await or nodejs (i recommend node for this type of stuff). Though the other bottleneck will be your HD in this case. When your HDs see 200000 IO calls the HD's themselves will start serializing the requests. Also with such short message flight time in the wire is super short. Likely your IO is spending a good chunk of time writing messages to program memory as well. The most fair test is 200000 IO calls to multiple external services that can handle such volume. Don't touch your HD. Just focus tests on services that will not block or serialize IO. >multiprocessing was able to parallelize that unambiguously. No dude. Wrong again. Have you noticed that you're able to create more processes than you have cores? Have you ever thought about all the processes that are running on your OS? Likely more total processes then you have cores. How does this happen? Because not all processes are parallel. For OS threads and for all processes your OS decides whether they will be parallel or whether they will not. I know of no API that gives you direct control over which Core to execute your thread/process on. The only difference between threads and processes is that threads have shared memory with other threads and are less expensive to instantiate in terms of time and memory and processes have their own memory space and are therefore more "expensive". Python is unique in the sense that threads are not parallel on compute due to the GIL, but this is only true for python and python-like languages. How's that for a wall of text? Hope you learned something junior.

jacobwilliamroy 6y ago
I did the benchmark and multiprocessing achieved higher bandwidth than multithreading. 8 times higher. This was just reading from disk, so no hashing step at all. Didn't even write to disk. I think you might not be so knowledgable about concurrency.
- Geminidog 6y ago
  Did you read my full post? I don't think you did. Let me quote it: >To make it a fair test and to see the benefits of python threads over processes you need to get rid of the hash operation. Maybe make your python program write a copy the file to another file. Then try to make everything concurrent. By everything I mean for every single file, fire off a new thread and do the same for processes. >Though the other bottleneck will be your HD in this case. When your HDs see 200000 IO calls the HD's themselves will start serializing the requests. Also with such short message flight time in the wire is super short. Likely your IO is spending a good chunk of time writing messages to program memory as well. >The most fair test is 200000 IO calls to multiple external services that can handle such volume. Don't touch your HD. Just focus tests on services that will not block or serialize IO. I'm very very knowledgeable about concurrency. The problem is you. You failed to read my post and the caveats of all the details I mentioned. You failed to adjust the test fully to make it fair. You just overall failed and you don't know much. Let me explain why you failed. Like I quoted above. HD IO has very very short time of flight meaning that the actual operation of compute (writing the data to program memory) takes much more time. Your threads aren't parallelizing that. Like I said before THE ONLY DIFFERENCE in processes and threads is that python threads DO NOT parallelize compute and processes are MORE EXPENSIVE to spawn in terms of memory space. SO if you logically parallelize off of a fix number of threads/processes what do you think will occur? Of course processes will be faster. I told you to adjust the test to spawn a new thread/process for every IO call to see the benefits. Because threads are cheaper in memory to spawn it will be able to spawn MANY more threads than it can processes. So if your operation is IO bound what you typically did with like 10 processes can likely be done with 1000 threads. <--- That is the difference. You said you have 2000000 files? That means 2000000 threads/processes must be spawned for a fair test. And if you can, try to hit some server on the internet that can handle that many requests to fully see parallelization of IO because time of flight is longer over the internet. Additionally your HDD's themselves can't parallelize 2000000 requests so in the end all your threads/processes will be bottlenecked by the HD itself NOT by IO. That's why I told you to hit something over the internet that can handle 2000000 IO calls. I mentioned All of this in my original response and you failed to understand. Now do you Get it? I actually am trying to teach you something and you have the balls to tell me that I don't know about concurrency. It's quite obvious you have no idea what you're talking about.