3 ms·
This was sort of true when computers had few CPUs/cores. It didn't make much of an efficiency difference, it was mostly a question of programming model. It work
by jake_morrison 6y ago
This was sort of true when computers had few CPUs/cores. It didn't make much of an efficiency difference, it was mostly a question of programming model. It works fine as long as the program is fundamentally I/O bound, but breaks down when it is doing actual work and becomes CPU bound.
I have certainly had some horrible experiences debugging multi-threaded C++ programs, and using non-blocking I/O made them safer and easier to reason about because only one thing happens at a time. The non-blocking I/O programming paradigm, however exposes unnecessary "plumbing" to the programmer.
What we really want is a concurrency model where we can handle each request as if it was the only thing happening.
To the extent that multiple requests need to share resources, we need ways to safely access state with concurrent access. That might be a relational database, mutexes, or software transactional memory.
I am currently in the middle of updating a large application written in Twisted Python. async/await and promises can certainly be complex. The Flask approach of using one thread per request is easier to understand. Everyone in the Javascript is in love with async/await, but I can see where that ends: just look at what Twisted was doing 10 years ago. And see that it basically failed due to complexity.
The model I like best is Erlang/Elixir. The runtime handles I/O using non-blocking I/O and threads to be efficient. The programmer works in terms of lightweight actors (called "processes", more like Java "green threads") that communicate using message passing. Each individual Erlang process only does one thing at a time.
As Joe Armstrong, one of Erlang's inventors, says: "We do not have ONE web-server handling two million sessions. We have two million webservers handling one session each." https://joearms.github.io/published/2016-03-13-Managing-two-million-webservers.html https://joearms.github.io/published/2016-03-13-Managing-two-...
Message passing handles concurrency without locks. If multiple client processes send update messages to a server, the server simply pulls them from its inbox in order and updates the shared state it is responsible for. The VM schedules processes preemptively, handling millions without breaking a sweat. It's safe, because one process can't mess with another process's memory. It is easy to reason about and debug.
It easily takes advantage of all the CPUs on the server. You can run a single VM instead of breaking up your application into multiple servers, each of which only uses one CPU (via containers or OS processes) and trying to coordinate between them. The VM has a built-in key/value store with efficient concurrency. It's like Redis but without the serialization overhead, so a read takes 1 microsecond.