6 ms·

Yes, and no. Concurrency and parallelism are important, but threads (lightweight shared memory processes) are not the only solution. Successful, popular concur

Yes, and no. Concurrency and parallelism are important, but threads (lightweight shared memory processes) are not the only solution. Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model. These platforms simplify application-level code by avoiding the issues of thread synchronization and contention in a shared memory space, and instead provide/require isolated processes to exploit CPU-level parallelism. The threading "isn't that bad" viewpoint generally comes from a limited understanding of the things that can go wrong - for instance, add "memory barriers" to the list of things to understand. There's a good explanation of the problems with a common Java idiom "double-checked locking" at https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedL... Of course, when things get complicated with lots of interactions between processes, or significant amounts of computation in a single-threaded process, some platforms require the user to manually yield, etc., basically trying to re-create preemptive threading provided by the OS.

dragonwriter 9y ago
> Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model. Erlang, the language, is threading model agnostic, but the main implementation has been, by default, M:N threaded on systems with multiple logical processors since, AFAICT, the R12B release in 2008. So characterizing it as single threaded is inaccurate.
- kbutler 9y ago
  Erlang the language provides single-threaded processes that do not have shared memory. The BEAM VM hosts Erlang processes in a multi-threaded single process. This is an implementation detail/performance optimization. The Erlang processes don't care if they are hosted in a single OS process or multiple processes or even multiple host machines. Erlang-the-platform simplifies application code by providing a single-threaded, share-nothing, async/non-blocking code model. (Edit: Yes, dfox, it's probably more accurate to say, Erlang provides a synchronous, blocking code model, since that's what the Erlang programmer sees)
  - dfox 9y ago
    From programmers view, synchronous blocking IO and IPC is the natural way. That it is mapped by the VM onto some asynchronous/event-triggered loop is also an implementation detail.
    - zzzcpan 9y ago
      Message passing in Erlang is asynchronous and this is one very important detail, that makes it good at concurrency. And equivalent to event driven programming to some extent.
      - dfox 9y ago
        Message passing in Erlang is asynchronous in the sense that messages can be queued, but from the point of view of the process receiving message is synchronous function call that happens on well defined place in the process, in contrast to truly asynchronous message receipt which would interrupt the process/start new thread at arbitrary places in code, which is the model used by some microkernels and similar systems (this can be emulated in userspace by starting background thread which runs some event loop, for example SDL, and optionally ALSA, uses this pattern for audio I/O).
  - dragonwriter 9y ago
    > Erlang the language provides single-threaded processes that do not have shared memory. Erlang processes share memory, and some message sends (local sends of binaries > 64 bytes in size) are implemented by sharing references to shared, reference-counted, objects. This is safe because (not considering the process dictionary, which isn't shared) Erlang processes have no mutable state, and all that is shared is immutable values, so if the implementation wanted to, all sends could be references to shared memory (for small objects, the indirection and reference counting overhead isn't worth the memory savings, so its not done.)
  - barrkel 9y ago
    The Erlang processes don't care if they are hosted in a single OS process or multiple processes or even multiple host machines. It's not possible to write a high-performance program without a good understanding of the costs of primitives. If something might vary in cost by several orders of magnitude, it's quite hard to design for. Network transparency has never been a good fundamental for well-performing application architecture. Networks are inherently far less reliable than shared memory, and have massively more latency, particularly on the tail end of the distribution.
slackingoff2017 9y ago
Nginx and Erlang single threaded? Uhhhhh, that's just not true. Nginx and Erlang most certainly use multiple threads by default. They both use thread pools. I'm not sure about Erlang but Nginx has a single master thread that does async select on the main socket. Once it receives a connection it hands off to worker threads. This is a traditional multithreading setup with control/worker threads. Node doesn't support threads because JavaScript doesn't. Performance would certainly be better for many workloads if it followed a traditional multithreading approach like Nginx. Async != Single threaded. They're not related concepts. You get the best performance using both non-blocking calls and multithreading , an approach used by Nginx and Netty. They both use nonblocking IO on the main thread and thread pools. Node requires you to manually yield by calling Response.end() because it doesn't support multithreading. This basically duplicates OS level pre-emption manually and is my single biggest criticism of node. The worker/master thread model allows the runtime to manage worker resources or just defer to the OS in a flexible way.
- kbutler 9y ago
  You are confusing threads with processes. Processes are isolated and do not share memory. Threads within a process share memory. For example, see https://msdn.microsoft.com/en-us/library/windows/desktop/ms684841(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/ms6... Nginx is single-threaded, multiple process: "Each worker process is single-threaded and runs independently" https://www.nginx.com/blog/inside-nginx-how-we-designed-for-performance-scale/ https://www.nginx.com/blog/inside-nginx-how-we-designed-for-... Erlang code is single threaded. Each Erlang "process" is an isolated, share-nothing single sequence of instructions, and you don't use semaphores, locks, or critical sections in writing Erlang code. The BEAM VM is implemented as a multithreaded process, but this is not exposed to the Erlang code, and Erlang code can be transparently distributed across multiple instances of the BEAM VM (processes) even on separate hosts. You're correct on node.js, and "basically trying to re-create preemptive threading provided by the OS."
  - slackingoff2017 9y ago
    Nginx recently implemented thread pools for blocking operations. The distinction between threads and processes depends on the OS. The "nothing shared" model used in Linux forking actually shares the underlying memory until the processes diverge (copy on write). The difference between this and threads is that threads expose the shared memory by default. IMO the difference between threads and processes is somewhat academic. I don't know much about BEAM
diek 9y ago
> Successful, popular concurrent platforms like [...] node.js, eschew threads in favor a single-threaded, async/non-blocking code model Except that Node (by way of libuv) uses thread pools internally for things that it deems to be CPU-intensive enough to need it, like crypto.
dfox 9y ago
In it's original application space the primary apeal of Erlang (as well as Ericsson's custom CPUs in their exchanges) was that user's interaction with the system (eg. one phone call) can be implemented by one thread/process running normal sequential synchronous code as opposed to as state in some complex asynchronous state machine (ie. what is done in node.js) that is orders of magnitude more complex to program and understand. Another solution to this problem involves inventing some monads-and-arrows-reinvented (async/await...) machinery that hides the state machine away. Multi-threaded programs have to be concerned with low-level details like memory ordering and barriers only when they do synchronization without using OS-provided utilities or when they implement synchronization wrong. pthread_mutex_lock() is both memory and optimization barrier and that is probably all one has to know for vast majority of MT code.
- kbutler 9y ago
  > when they implement synchronization wrong. Yes, but there are lots of ways to implement synchronization wrong because you don't understand memory barriers. For instance, Doug Lea's article linked above includes this very simple Java code, which breaks because it wasn't aware of memory barriers: // Broken multithreaded version // "Double-Checked Locking" idiom class Foo { private Helper helper = null; public Helper getHelper() { if (helper == null) synchronized(this) { if (helper == null) helper = new Helper(); } return helper; } // other functions and members... } [dfox: can't reply, guess we're nesting too deeply. Per the above, the simple rule has to be modified to "do not touch OR READ shared state outside of relevant critical section". Easy to make errors...]
  - dfox 9y ago
    There is simple rule for that: do not touch shared state outside of relevant critical section. To some extent the cited problem is unique to Java, because the whole monitor concept just creates confusion about what synchronized really means (and also creates significant overhead by making literally everything usable as mutex). Pretty common misconception is that synchronized is akin to some kind of read-write lock (in which case the code above would work as expected).
openasocket 9y ago
It all depends on what problem you're solving. Note that you're focusing on libraries and languages for writing servers. But what about, say, linear algebra routines? How would you implement BLAS in a performant manner without threads? Sometimes, threads are the preferable solution.
gozur88 9y ago
>Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model. Ugh. As someone suffering with a node.js project now I'm continually chafing under this limitation. It's great if you're making a simple CRUD website, but you'll be beating your head against the wall if you have a problem domain with centralized resource sharing that prevents you from spamming processes willy-nilly. And node.js code is ugly. >The threading "isn't that bad" viewpoint generally comes from a limited understanding of the things that can go wrong - for instance, add "memory barriers" to the list of things to understand. Actually, threading isn't very hard to do right. It's more a question of choosing a reasonable architecture. Sure, you can get yourself into a lot of trouble trying to interleave threads in an attempt to use multiple processors to speed up a single operation. Don't do that. >There's a good explanation of the problems with a common Java idiom "double-checked locking" at https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedL.. https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedL.... Yes, and if you read to the end of the article you'll see the problem it's addressing was fixed 12 years ago in Java 5. As someone who's implemented reliable, massively parallel threaded systems in Java, when I hear people say threading is too complicated to be used safely I feel like a blacksmith who's just been told horseshoes are impossible to make even though he's been making them his whole career.
- tracker1 9y ago
  As to the ugliness of node.js code, I find that if you're using a transpiler (babel) or the latest 8.x, that async/await with promises tends to clean things up a lot.
  - chrisco255 9y ago
    Yeah, the Koa2 framework for Node.js is a beautiful way to handle requests in my opinion and it's built on async/await functionality from the ground up.
- kbutler 9y ago
  Yeah, re-reading my comment above, it feels like I'm saying that single-threaded async is the one-true-way, and I don't think that. Both threaded and async/CSP models are useful, and both do have weaknesses. I've also implemented reliable, parallel, `threaded systems in Java. But I've also found defects in code that I thought had been thoroughly exercised, especially when it has moved to a new environment or new level of parallelism. There's often "one more thing" that can cause problems, and generally it can be resolved by one more concept or workaround (like adding "volatile" in Java 5). And I'd suggest Erlang/Go, which are designed for CSP-style concurrency, for cleaner code than node.js. JavaScript-based platforms often feel like, "Wow, you can even make JavaScript do X" rather than, "X is really natural in JavaScript."
hyperpape 9y ago
Double-checked locking has been solved in Java since 1.5, which arrived more than a decade ago. The article you link to even says it's going to be solved.
- paulddraper 9y ago
  "Solved" if you remember to mark the static as volatile. It still proves the OP's point that this stuff can be come complicated.
  - hyperpape 9y ago
    That doesn't seem that bad to me. It's an explicit, unambiguous recommendation, one that even be recognized by static analysis. It's not much worse than an API that requires that you call some particular method--the only part that's bad is that you can fail to notice the problem because the code will usually work. What I worry about are things that can't be easily caught. I trust myself to write embarrassingly parallel code, and I've written some code that has real risks of deadlock. But shared data structures that have complex locking schemes still scare me.
chaotic-good 9y ago
> Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model. All these platforms have some problems. E.g. Nginx is PITA if your processes need to communicate with each other. Erlang/OTP has this nasty mailbox problem (O(N^2) behavior of the selective receive). The cooperative multitasking is a piece of Victorian-era technology that is so painfully bad for various reasons, but people still tend to believe that is solves something. On the other hand, modern operating systems are awesome. The thread schedules are awesome. Thread schedulers can dynamically adjust thread priorities depending on the load and solve some nasty synchronization problems (like priority inversion or starvation) for you. It's not 1995 and OS schedulers can switch threads in O(1) and most server apps can use a thread per connection approach without any problems. The only thing you need to get right is a synchronization, but there are a lot of tools that can help (like Thread Sanitizer). It's much harder to get the synchronization right with cooperative multitasking (good luck finding that priority inversion on implicit lock caused by dumb channel use pattern) than with normal threads.

dragonwriter 9y ago

> Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model. Erlang, the language, is threading model agnostic, but the main implementation has been, by default, M:N threaded on systems with multiple logical processors since, AFAICT, the R12B release in 2008. So characterizing it as single threaded is inaccurate.

kbutler 9y ago
Erlang the language provides single-threaded processes that do not have shared memory. The BEAM VM hosts Erlang processes in a multi-threaded single process. This is an implementation detail/performance optimization. The Erlang processes don't care if they are hosted in a single OS process or multiple processes or even multiple host machines. Erlang-the-platform simplifies application code by providing a single-threaded, share-nothing, async/non-blocking code model. (Edit: Yes, dfox, it's probably more accurate to say, Erlang provides a synchronous, blocking code model, since that's what the Erlang programmer sees)
- dfox 9y ago
  From programmers view, synchronous blocking IO and IPC is the natural way. That it is mapped by the VM onto some asynchronous/event-triggered loop is also an implementation detail.
  - zzzcpan 9y ago
    Message passing in Erlang is asynchronous and this is one very important detail, that makes it good at concurrency. And equivalent to event driven programming to some extent.
    - dfox 9y ago
      Message passing in Erlang is asynchronous in the sense that messages can be queued, but from the point of view of the process receiving message is synchronous function call that happens on well defined place in the process, in contrast to truly asynchronous message receipt which would interrupt the process/start new thread at arbitrary places in code, which is the model used by some microkernels and similar systems (this can be emulated in userspace by starting background thread which runs some event loop, for example SDL, and optionally ALSA, uses this pattern for audio I/O).
- dragonwriter 9y ago
  > Erlang the language provides single-threaded processes that do not have shared memory. Erlang processes share memory, and some message sends (local sends of binaries > 64 bytes in size) are implemented by sharing references to shared, reference-counted, objects. This is safe because (not considering the process dictionary, which isn't shared) Erlang processes have no mutable state, and all that is shared is immutable values, so if the implementation wanted to, all sends could be references to shared memory (for small objects, the indirection and reference counting overhead isn't worth the memory savings, so its not done.)
- barrkel 9y ago
  The Erlang processes don't care if they are hosted in a single OS process or multiple processes or even multiple host machines. It's not possible to write a high-performance program without a good understanding of the costs of primitives. If something might vary in cost by several orders of magnitude, it's quite hard to design for. Network transparency has never been a good fundamental for well-performing application architecture. Networks are inherently far less reliable than shared memory, and have massively more latency, particularly on the tail end of the distribution.

slackingoff2017 9y ago

Nginx and Erlang single threaded? Uhhhhh, that's just not true. Nginx and Erlang most certainly use multiple threads by default. They both use thread pools. I'm not sure about Erlang but Nginx has a single master thread that does async select on the main socket. Once it receives a connection it hands off to worker threads. This is a traditional multithreading setup with control/worker threads. Node doesn't support threads because JavaScript doesn't. Performance would certainly be better for many workloads if it followed a traditional multithreading approach like Nginx. Async != Single threaded. They're not related concepts. You get the best performance using both non-blocking calls and multithreading , an approach used by Nginx and Netty. They both use nonblocking IO on the main thread and thread pools. Node requires you to manually yield by calling Response.end() because it doesn't support multithreading. This basically duplicates OS level pre-emption manually and is my single biggest criticism of node. The worker/master thread model allows the runtime to manage worker resources or just defer to the OS in a flexible way.

kbutler 9y ago
You are confusing threads with processes. Processes are isolated and do not share memory. Threads within a process share memory. For example, see https://msdn.microsoft.com/en-us/library/windows/desktop/ms684841(v=vs.85).aspx https://msdn.microsoft.com/en-us/library/windows/desktop/ms6... Nginx is single-threaded, multiple process: "Each worker process is single-threaded and runs independently" https://www.nginx.com/blog/inside-nginx-how-we-designed-for-performance-scale/ https://www.nginx.com/blog/inside-nginx-how-we-designed-for-... Erlang code is single threaded. Each Erlang "process" is an isolated, share-nothing single sequence of instructions, and you don't use semaphores, locks, or critical sections in writing Erlang code. The BEAM VM is implemented as a multithreaded process, but this is not exposed to the Erlang code, and Erlang code can be transparently distributed across multiple instances of the BEAM VM (processes) even on separate hosts. You're correct on node.js, and "basically trying to re-create preemptive threading provided by the OS."
- slackingoff2017 9y ago
  Nginx recently implemented thread pools for blocking operations. The distinction between threads and processes depends on the OS. The "nothing shared" model used in Linux forking actually shares the underlying memory until the processes diverge (copy on write). The difference between this and threads is that threads expose the shared memory by default. IMO the difference between threads and processes is somewhat academic. I don't know much about BEAM

diek 9y ago

> Successful, popular concurrent platforms like [...] node.js, eschew threads in favor a single-threaded, async/non-blocking code model Except that Node (by way of libuv) uses thread pools internally for things that it deems to be CPU-intensive enough to need it, like crypto.

dfox 9y ago

In it's original application space the primary apeal of Erlang (as well as Ericsson's custom CPUs in their exchanges) was that user's interaction with the system (eg. one phone call) can be implemented by one thread/process running normal sequential synchronous code as opposed to as state in some complex asynchronous state machine (ie. what is done in node.js) that is orders of magnitude more complex to program and understand. Another solution to this problem involves inventing some monads-and-arrows-reinvented (async/await...) machinery that hides the state machine away. Multi-threaded programs have to be concerned with low-level details like memory ordering and barriers only when they do synchronization without using OS-provided utilities or when they implement synchronization wrong. pthread_mutex_lock() is both memory and optimization barrier and that is probably all one has to know for vast majority of MT code.

kbutler 9y ago
> when they implement synchronization wrong. Yes, but there are lots of ways to implement synchronization wrong because you don't understand memory barriers. For instance, Doug Lea's article linked above includes this very simple Java code, which breaks because it wasn't aware of memory barriers: // Broken multithreaded version // "Double-Checked Locking" idiom class Foo { private Helper helper = null; public Helper getHelper() { if (helper == null) synchronized(this) { if (helper == null) helper = new Helper(); } return helper; } // other functions and members... } [dfox: can't reply, guess we're nesting too deeply. Per the above, the simple rule has to be modified to "do not touch OR READ shared state outside of relevant critical section". Easy to make errors...]
- dfox 9y ago
  There is simple rule for that: do not touch shared state outside of relevant critical section. To some extent the cited problem is unique to Java, because the whole monitor concept just creates confusion about what synchronized really means (and also creates significant overhead by making literally everything usable as mutex). Pretty common misconception is that synchronized is akin to some kind of read-write lock (in which case the code above would work as expected).

openasocket 9y ago

It all depends on what problem you're solving. Note that you're focusing on libraries and languages for writing servers. But what about, say, linear algebra routines? How would you implement BLAS in a performant manner without threads? Sometimes, threads are the preferable solution.

gozur88 9y ago

>Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model. Ugh. As someone suffering with a node.js project now I'm continually chafing under this limitation. It's great if you're making a simple CRUD website, but you'll be beating your head against the wall if you have a problem domain with centralized resource sharing that prevents you from spamming processes willy-nilly. And node.js code is ugly. >The threading "isn't that bad" viewpoint generally comes from a limited understanding of the things that can go wrong - for instance, add "memory barriers" to the list of things to understand. Actually, threading isn't very hard to do right. It's more a question of choosing a reasonable architecture. Sure, you can get yourself into a lot of trouble trying to interleave threads in an attempt to use multiple processors to speed up a single operation. Don't do that. >There's a good explanation of the problems with a common Java idiom "double-checked locking" at https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedL.. https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedL.... Yes, and if you read to the end of the article you'll see the problem it's addressing was fixed 12 years ago in Java 5. As someone who's implemented reliable, massively parallel threaded systems in Java, when I hear people say threading is too complicated to be used safely I feel like a blacksmith who's just been told horseshoes are impossible to make even though he's been making them his whole career.

tracker1 9y ago
As to the ugliness of node.js code, I find that if you're using a transpiler (babel) or the latest 8.x, that async/await with promises tends to clean things up a lot.
- chrisco255 9y ago
  Yeah, the Koa2 framework for Node.js is a beautiful way to handle requests in my opinion and it's built on async/await functionality from the ground up.
kbutler 9y ago
Yeah, re-reading my comment above, it feels like I'm saying that single-threaded async is the one-true-way, and I don't think that. Both threaded and async/CSP models are useful, and both do have weaknesses. I've also implemented reliable, parallel, `threaded systems in Java. But I've also found defects in code that I thought had been thoroughly exercised, especially when it has moved to a new environment or new level of parallelism. There's often "one more thing" that can cause problems, and generally it can be resolved by one more concept or workaround (like adding "volatile" in Java 5). And I'd suggest Erlang/Go, which are designed for CSP-style concurrency, for cleaner code than node.js. JavaScript-based platforms often feel like, "Wow, you can even make JavaScript do X" rather than, "X is really natural in JavaScript."

hyperpape 9y ago

Double-checked locking has been solved in Java since 1.5, which arrived more than a decade ago. The article you link to even says it's going to be solved.

paulddraper 9y ago
"Solved" if you remember to mark the static as volatile. It still proves the OP's point that this stuff can be come complicated.
- hyperpape 9y ago
  That doesn't seem that bad to me. It's an explicit, unambiguous recommendation, one that even be recognized by static analysis. It's not much worse than an API that requires that you call some particular method--the only part that's bad is that you can fail to notice the problem because the code will usually work. What I worry about are things that can't be easily caught. I trust myself to write embarrassingly parallel code, and I've written some code that has real risks of deadlock. But shared data structures that have complex locking schemes still scare me.

chaotic-good 9y ago

> Successful, popular concurrent platforms like Erlang/OTP, Nginx, and node.js, eschew threads in favor a single-threaded, async/non-blocking code model. All these platforms have some problems. E.g. Nginx is PITA if your processes need to communicate with each other. Erlang/OTP has this nasty mailbox problem (O(N^2) behavior of the selective receive). The cooperative multitasking is a piece of Victorian-era technology that is so painfully bad for various reasons, but people still tend to believe that is solves something. On the other hand, modern operating systems are awesome. The thread schedules are awesome. Thread schedulers can dynamically adjust thread priorities depending on the load and solve some nasty synchronization problems (like priority inversion or starvation) for you. It's not 1995 and OS schedulers can switch threads in O(1) and most server apps can use a thread per connection approach without any problems. The only thing you need to get right is a synchronization, but there are a lot of tools that can help (like Thread Sanitizer). It's much harder to get the synchronization right with cooperative multitasking (good luck finding that priority inversion on implicit lock caused by dumb channel use pattern) than with normal threads.