4 ms·
I think this is rather confusing and it introduces un-necessary abstractions -- "working" and "waiting". It is easier to think of it as: * concurrency -- a pr
by rdtsc 13y ago
I think this is rather confusing and it introduces un-necessary abstractions -- "working" and "waiting".
It is easier to think of it as:
* concurrency -- a property of the algorithm or the business process where multiple things can be can happen at the same time. Think about mergesort for example, you can sort the left and the right parts of the list at the same time. There is no guarantee that when the program executes concurrent things will actually execute at the same time. Or think of fetching web pages from multiple websites with no need to wait between them.
* parallelism -- a property of the execution environment. Based on the specific hardware, and OS facilities, some of the concurrent things can actually happen at the same time. Can fetch the web pages from 100 sites at the same time. Will actually sort both sides of the array on two separate CPU cores. And so on.
It is as simple as that. There is some fuzziness around the two. Like saying can one have parallelism without concurrency. Maybe your compiler noticed how it could transform your for loop into a SIMD operation. Ok, is that concurrency and parallelism or is it just parallelism. I don't really bother being that pedantic at that point. Is it concurrency only if user specifically uses provided language concurrency primitives or does "implicit" (let compiler figure it out) concurrency also count kind of stuff.
But, wait, that's not all. There is another axis of comparison that is useful in practice, and that is CPU vs I/O.
* CPU : Mergesort example above would be an example of mostly CPU concurrency and parallelism. You can spawn two threads to make it explicit and if you have more than one core, you might get some CPU parallelism.
* I/O : You an fetch 2 pages at once using a thread per page and your socket layer works properly you might get good parallelism.
I/O is where the discussion of the GIL comes in. Python has great I/O parallelism. GIL doesn't really matter much for it. I have written a spyder like I described and it was very performant, very nice, just using plain on GIL-ified threads.
Anyway it is important to keep these concepts clear to be aware of them. These are important today and I feel they are not taught well enough in academia (not in this explicit, comparative fashion).
- derefr 13y agoAn interesting property of CPU vs. I/O concurrency, to me, is that while I/O-heavy concurrency works fine with cooperative or reduction-counting scheduling, CPU-heavy concurrency basically requires a preemptive scheduling approach (i.e. multiple native threads.) A lot of people would assume that on a single-core machine, you could get away with running, for example, Erlang with only a single native scheduler-thread for its tasks. This would be true until the first time you called a C-FFI function that took a full CPU-second to return. Then all the other tasks in your system would wake up wondering what year it is and how their beards grew so long. Erlang added "dirty schedulers" (basically, extra scheduler-threads that will get spawned above-and-beyond the default one-per-core-with-CPU-affinity set) precisely so the preemption of CPU-heavy tasks could be pushed off into the OS's capable hands.
- rdtsc 13y ago> This would be true until the first time you called a C-FFI function that took a full CPU-second to return. As you said, up until last release that would be true in general. Even if you had 16 CPUs running C NIFs inside of it that block for longer than 1ms at a time would probably be a bad idea. Word of warning, think really well about compiling in and sticking native code (NIF) drivers in the middle of the Erlang VM. You lose reliability, fault tolerance, predictable low latency response. Good examples of NIFs could be computing a hash function, or something similar that would be constant time and easily tested. Not something like getting a piece of data from a database.
- derefr 13y agoI think the best case for dirty schedulers in Erlang isn't for running CPU-intensive tasks within the same VM that's doing your IO, though; it's for running a separate Erlang node, on a separate machine, and sending it CPU-heavy work to do over the distribution protocol. Effectively, an isolated Erlang node + dirty schedulers + NIFs is a souped-up, easier-to-code-for C node.
- _yosefk 13y agoWell, "working vs waiting" is the same as "CPU vs I/O", so we're saying the same thing here. As to your "concurrency vs parallelism" distinction - it's one valid one, and pretty widespread; I think it misses a lot of important points though, and I wrote about it here: http://www.yosefk.com/blog/parallelism-and-concurrency-need-different-tools.html http://www.yosefk.com/blog/parallelism-and-concurrency-need-... One point from there is, a parallel mergesort is conflict-free while say a concurrent DB-backed web app has a lot of opportunities for conflicts (2 users want to take the same username - who wins? etc. etc.) I argue that the many differences between such systems result in rather different tools being suitable to build them.