4 ms·

I have always had a hard time with this parallel / concurrent distinction. I think it’s because of how many layers of abstraction the machine has. Many thread

I have always had a hard time with this parallel / concurrent distinction. I think it’s because of how many layers of abstraction the machine has. Many threads per core. Many cores per package. Multiple packages on a motherboard. Is there a practical distinction? It feels like there isn’t. It’s not as if a two CPU machine can run two Python processes “in parallel” as opposed to “concurrently”. I have 100 other processes running too. It’s all concurrency, and never parallel. Perhaps the author and others are distinguishing between concurrency that can happen at the granularity of any instruction (e.g. threading) compared to cooperative yielding that only happens when await is called (e.g. asyncio.)

Izkata 3y ago
> It’s not as if a two CPU machine can run two Python processes “in parallel” as opposed to “concurrently”. I have 100 other processes running too. It’s all concurrency, and never parallel. If they're both CPU-bound they probably can. Most of the time those 100 other processes are probably sleeping and not actually using the CPU (unless you're at 100% CPU usage all the time? My computer tends to sit at around 5-15% usage)
josephg 3y ago
> Is there a practical distinction? I’m not sure if this will help, but I think of it like - parallelism is running the same task in multiple threads / on multiple cores at once. And concurrency is “concurrently” running different tasks on one core. Nodejs is a purely single threaded runtime. Let’s say you have a web server in node processing http requests. - Your program implements concurrency by yielding back to the event loop instead of blocking. Request 1 comes in, and then sends a query to the database. While we wait for the database to respond to the query, we can process request 2. - You usually implement parallelism with nodejs by running multiple instances of your nodejs server process across all your cores. (Then use a program like nginx to load balance requests across all the node processes.) The GPU might be a better example. When rendering a frame, the graphics card runs a small program (called a fragment shader) for every pixel on the screen. A 4090 has ~16000 cores which can all run the same fragment shader on different pixels at the same time. This is called parallelism, not concurrency. Concurrency would be if you had two video games open at once and your graphics card was quickly swapping between rendering frames for game A and game B.
plonk 3y ago
My view: “concurrent” just means that there can be a point in time when two tasks/functions/coroutines/whatever have started but haven’t finished. “Parallel” means that they actually run at the same time, e.g. on different CPU cores. Concurrent is more general.

Izkata 3y ago

> It’s not as if a two CPU machine can run two Python processes “in parallel” as opposed to “concurrently”. I have 100 other processes running too. It’s all concurrency, and never parallel. If they're both CPU-bound they probably can. Most of the time those 100 other processes are probably sleeping and not actually using the CPU (unless you're at 100% CPU usage all the time? My computer tends to sit at around 5-15% usage)

josephg 3y ago

> Is there a practical distinction? I’m not sure if this will help, but I think of it like - parallelism is running the same task in multiple threads / on multiple cores at once. And concurrency is “concurrently” running different tasks on one core. Nodejs is a purely single threaded runtime. Let’s say you have a web server in node processing http requests. - Your program implements concurrency by yielding back to the event loop instead of blocking. Request 1 comes in, and then sends a query to the database. While we wait for the database to respond to the query, we can process request 2. - You usually implement parallelism with nodejs by running multiple instances of your nodejs server process across all your cores. (Then use a program like nginx to load balance requests across all the node processes.) The GPU might be a better example. When rendering a frame, the graphics card runs a small program (called a fragment shader) for every pixel on the screen. A 4090 has ~16000 cores which can all run the same fragment shader on different pixels at the same time. This is called parallelism, not concurrency. Concurrency would be if you had two video games open at once and your graphics card was quickly swapping between rendering frames for game A and game B.

plonk 3y ago

My view: “concurrent” just means that there can be a point in time when two tasks/functions/coroutines/whatever have started but haven’t finished. “Parallel” means that they actually run at the same time, e.g. on different CPU cores. Concurrent is more general.