5 ms·

I don't write web applications, and I don't use much JavaScript, so it's very possible that I don't properly understand the motivation for this blog post. Howev

I don't write web applications, and I don't use much JavaScript, so it's very possible that I don't properly understand the motivation for this blog post. However, as others have said there is a difference between some inconvenient syntax in JS, and the fundamental model of non-blocking I/O. What I do write is lots of C/C++ client/server applications for HPC/HFT/DC workloads where speed both in req/sec (throughput) and speed in min/avg/max(secs/req) (latency) matters. In these environments I almost exclusively use non-blocking I/O. There are several reasons: 1) Threads are not free. Even if you use a thread-pool to avoid spin up costs, context switching overhead matters. Every time you call blocking I/O, you make sure that the kernel will wake up, schedule another thread, and do anything else that it decides to do. Waste time that you could have used to do useful work. Non-blocking I/O puts you in charge of your own "thread scheduler". Your "threads" are functions, they are "cooperatively scheduled" and you can make full use of every cycle that you get. 2) Programming with threads is hard. Trust me. If you think it's easy, or I'm soft, you haven't done it enough. At some point you will need shared state across those threads. And then you'll need locking and unlocking. (also Mutexs are slooooowww) And then you'll need to handle error cases, and you'll need to make sure that all the unlocking is done right in all of the right places. And then you'll need signaling between your threads. And you'll need semaphores or similar. And 3 months down the line, you're thinking to yourself, when a foo exception causes a bar signal, will a baz handler deadlock? Will it make progress? Humans just aren't designed to reason about this sort of thing. With a single threaded, non-blocking design, it's really easy to reason about exactly what is happening with all of your state. Debugging is obvious and straightforward. This is necessary if you're like me and don't write perfect code first time. There's only ever one function accessing shared state at one time. The "scheduler" is working for you, not against you. If you write your code simply, cleanly and efficiently, you'd be amazed how much work a modern CPU can really do. Honestly, once you've saturated a 10G NIC what more do you want to do? 3) If you buy into the non-blocking design, then, as long as you only use 1 process/thread per core, almost anything a thread can do, a process can do better. Threads have no memory protection, anything you touch probably belongs to some other thread and you're inviting subtle bugs. Processes have memory protection by default if you want to share things you can do it explicitly via safe mechanisms (shared memory rings, pipes, IPC etc). Shared memory rings are (can be) so fast that data is more or less local so if you want to use shared state from a TCP connection or whatever, you can always "dispatch" work to another process to do it for you. You get the benefits of many cores working for you as well as a clean and obvious programming model. Ultimately, if the question is one of syntax, then I'd happily believe that JS has some ugly syntax for doing these things, but if the question is one of design, then you should think really really hard before deciding that a threaded model is the correct one for you.

JoeAltmaier 12y ago
Of course it all depends upon the application. Processes are very heavy weight constructs. If you have a variable number of low-latency or lengthy tasks to do (streaming media, data reformatting, responding to external events) then threads are a good fit. Encapsulation is your friends. If each thread deals with a non-overlapping (set of) object(s), then many of the issues are gone. What is left is messaging between a thread and the process, which can be done using a thread-safe queue.
- deadgrey19 12y ago
  Hmm. I don't think I made myself very clear. I failed to mention that I'm advocating 1 process per core, not hundreds (thousands) of processes. Over subscribing processes to cores has the same effect of oversubscribing threads to cores, which is suboptimal scheduling. Furthermore, this model allows you to easily pin work to cores (using process affinity) and to get all the juicy benefits of using cooperative scheduling inside your processes. I think the view that processes are "heavy weight" is a dated one. From an OS point of view, processes are pretty much the same amount of "work" as thread. Each has a context, each needs to be scheduled. Processes do have some extra state (notably the TLB context) but modern machines are very good switching these. Spinning up processes is somewhat more expensive than spinning up threads, but you really shouldn't be doing either on the critical path. I agree that everything is context specific, although my main application area is really low latency scenarios (handfuls of microseconds) and techniques that work well there tend to port well to slower situations pretty easily (at least in my experience). I disagree that encapsulation and non-overlapping objects will save you from threading nightmares. Every design starts out with clean boundaries and beautiful abstractions. Every design ends up in spaghetti soup. It's just a matter of how long it takes to get there.
  - JoeAltmaier 12y ago
    Processes are heavy weight. They use lots of memory (megabytes) each, and they are very, very slow to switch between (tlb cache blown). Threads are better by an order of magnitude or two, for switching latency. Encapsulation saves you from threading nightmares. Your spaghetti notwithstanding. I've done this over 20 years and several startups, writing entire application environments on everything from embedded to desktop to server, and it works fine. Try it.

JoeAltmaier 12y ago

Of course it all depends upon the application. Processes are very heavy weight constructs. If you have a variable number of low-latency or lengthy tasks to do (streaming media, data reformatting, responding to external events) then threads are a good fit. Encapsulation is your friends. If each thread deals with a non-overlapping (set of) object(s), then many of the issues are gone. What is left is messaging between a thread and the process, which can be done using a thread-safe queue.

deadgrey19 12y ago
Hmm. I don't think I made myself very clear. I failed to mention that I'm advocating 1 process per core, not hundreds (thousands) of processes. Over subscribing processes to cores has the same effect of oversubscribing threads to cores, which is suboptimal scheduling. Furthermore, this model allows you to easily pin work to cores (using process affinity) and to get all the juicy benefits of using cooperative scheduling inside your processes. I think the view that processes are "heavy weight" is a dated one. From an OS point of view, processes are pretty much the same amount of "work" as thread. Each has a context, each needs to be scheduled. Processes do have some extra state (notably the TLB context) but modern machines are very good switching these. Spinning up processes is somewhat more expensive than spinning up threads, but you really shouldn't be doing either on the critical path. I agree that everything is context specific, although my main application area is really low latency scenarios (handfuls of microseconds) and techniques that work well there tend to port well to slower situations pretty easily (at least in my experience). I disagree that encapsulation and non-overlapping objects will save you from threading nightmares. Every design starts out with clean boundaries and beautiful abstractions. Every design ends up in spaghetti soup. It's just a matter of how long it takes to get there.
- JoeAltmaier 12y ago
  Processes are heavy weight. They use lots of memory (megabytes) each, and they are very, very slow to switch between (tlb cache blown). Threads are better by an order of magnitude or two, for switching latency. Encapsulation saves you from threading nightmares. Your spaghetti notwithstanding. I've done this over 20 years and several startups, writing entire application environments on everything from embedded to desktop to server, and it works fine. Try it.