4 ms·

The idea espoused in this blog post, that > if you have N logical threads concurrently executing a routine with Y yield points, then there are NY possible exe

The idea espoused in this blog post, that > if you have N logical threads concurrently executing a routine with Y yield points, then there are NY possible execution orders that you have to hold in your head is actively harmful to software maintainability. Concurrency problems don't disappear when you make your yield points explicit. Look: in traditional multi-threaded programs, we protect shared data using locks. If you avoid explicit locks and instead rely on complete knowledge of all yield points (i.e., all possible execution orders) to ensure that data races do not happen, then you've just created a ticking time-bomb: as soon as you add a new yield point, you invalidate your safety assumptions. Traditional lock-based preemptive multi-threaded code isn't susceptible to this problem: it already embeds maximally pessimistic assumptions about execution order, so adding a new preemption point cannot hurt anything. Of course, you can use mutexes with explicit yield points too, but nobody does: the perception is that cooperative multitasking (or promises or whatever) frees you from having to worry about all that hard, nasty multi-threaded stuff you hated in your CS classes. But you haven't really escaped. Those dining philosophers are still there, and now they're angry. The article claims that yield-based programming is easier because the fewer the total number of yield points, the less mental state a programmer needs to maintain. I don't think this argument is correct: in lock-based programming, we need to keep _zero_ preemption points in mind, because we assume every instruction is a yield point. Instead of thinking about NY program interleavings, we think about how many locks we hold. I bet we have fewer locks than you have yields. To put it another way, the composition properties of locks are much saner than the composition properties of safety-through-controlling-yield. I believe that we got multithreaded programming basically right a long time ago, and that improvement now rests on approaches like reducing mutable shared state, automated thread-safety analysis, and software transactional memory. Encouraging developers to sprinkle "async" and "await" everywhere is a step backward in performance, readability, and robustness.

vomjom 10y ago
It's not clear what you're suggesting as an alternative. My understanding is that you're suggesting thread-per-request, which has many known flaws. There are three approaches to serving requests: 1. Thread-per-request. This is a simple model. You have a fixed-size thread pool of size N, and once you hit that limit, you can't serve anymore requests. Thread-per-request has several sources of overhead, which is why people recommend against it: thread limits, per-thread stack memory usage, and context switching. 2. Coroutine style handling with cooperative scheduling at synchronization points (locks, I/O). This is how Go handles requests. 3. Asynchronous request handling. You still have a fixed-size thread pool handling requests, but you no longer limit the number of simultaneous requests with the size of that thread pool. There are several different styles of async request handling: callbacks, async/await, and futures. #2 and #3 are more common these days because they don't suffer from the many drawbacks of the thread-per-request model, although both suffer from some understandability issues.
- quotemstr 10y ago
  Those options aren't as distinct as you might imagine. Would calling it fiber-per-request make you happy? (By the way: most of the time, a plain-old-boring thread-per-request is just fine, because most of the time, you're not writing high-scale software. If you have at most two dozen concurrent tasks, you're wasting your time worrying about the overhead of plain old pthread_t.) I'm using a much more expansive definition of "thread" than you are. Sure, in the right situation, maybe M:N threading, or full green threads, or whatever is the right implementation strategy. There's no reason that green threading has to involve the use of explicit "async" and "await" keywords, and it's these keywords that I consider silly.
  - vomjom 10y ago
    (I agree that thread-per-request works just fine in the majority of cases, but it's still worthwhile to write about the cases where it doesn't work.) Responding to your original post: you argue that async/await intends to solve the problem of data races. That's not why people use it, nor does it tackle that problem at all (you still need locks around shared data). It only tries to solve the issue of highly-concurrent servers, where requests are bound by some resource that a request-handling threads have to wait for the result of (typically I/O). Coroutines/fibers are not an alternative to async servers, because they need primitives that are either baked into the language or the OS itself to work well.
    - cderwin 10y ago
      Please correct me if I'm wrong, but doesn't asyncio in the form of async/await (or any other ways to explicitly denote context switches) solve the problem of data races in that per-thread data structures can be operated on atomically by different coroutines? My understanding is that unless data structures are shared with another thread, you don't usually need locks for shared data.
      - omribahumi 10y ago
        I think that the biggest argument against it is code changes. Think about a code change that adds an additional yield point without proper locking. Has any language tackled this with lazy locking? i.e. lock only on yield. Maybe this could even be done in compile time
kmike84 10y ago
In his talk about threads (https://www.youtube.com/watch?v=Bv25Dwe84g0 https://www.youtube.com/watch?v=Bv25Dwe84g0) Raymond Hettinger made a point: when you have a codebase with many locks the result of composition is often a sequential program.
Arnt 10y ago
I think the general idea is that if you make your yield points explicit, you see them yourself, and if you generally try for 100% unit test coverage, you end up nudging yourself towards fewer yield points, a simpler state machine, and better karma overall. The complexity you see affects yourself more than the complexity you don't. I rather agree. FWIW my friend Abhijit Menon-Sen wrote a blog post on the matter last year, about some code with excellent test coverage and explicit yield points: http://toroid.org/callback-heaven http://toroid.org/callback-heaven
lmm 10y ago
> Look: in traditional multi-threaded programs, we protect shared data using locks. If you avoid explicit locks and instead rely on complete knowledge of all yield points (i.e., all possible execution orders) to ensure that data races do not happen, then you've just created a ticking time-bomb: as soon as you add a new yield point, you invalidate your safety assumptions. > Traditional lock-based preemptive multi-threaded code isn't susceptible to this problem: it already embeds maximally pessimistic assumptions about execution order, so adding a new preemption point cannot hurt anything. You get an equal and opposite problem: whenever you add one more lock, you invalidate your liveness assumptions. > The article claims that yield-based programming is easier because the fewer the total number of yield points, the less mental state a programmer needs to maintain. I don't think this argument is correct: in lock-based programming, we need to keep _zero_ preemption points in mind, because we assume every instruction is a yield point. Instead of thinking about NY program interleavings, we think about how many locks we hold. I bet we have fewer locks than you have yields. I'll take that bet. You really don't have to yield very often - only when making a network request, and perhaps not even for that in the case of a fast local network. Whereas you have to lock every piece of state that you have.
- IgorPartola 10y ago
  Every piece of shared state. In the case of something like a web server, you only need one lock: when the connection is being accepted and handed off to a worker thread. How many reads and writes does your web server perform?
- marcosdumay 10y ago
  > Whereas you have to lock every piece of state that you have. You need to lock every piece of shared state you have. Where "shared" means stuff that many threads must communicate among themselves. One tends to keep the number of that kind of state low, really low. When zero is not possible, the most common number by a wide margin is one¹. If you have more than 1, they are normally completely independent pieces of state that will not be used at the same time. If you have more than 1, and they are not independent, the code is either the result of at least one PHD thesis, or it does not work (or, often, both). I bet you do network requests more than once on your code. 1 - The size of the shared state does not matter, so it's often one really big state.
nhumrich 10y ago
Your saying to say that async will still have race conditions, but I disagree with that premise. On async, context switching onlykhappens on an "await", which means race conditions won't happen. Race conditions typically happen when you have a shared global state and you read and write to it in two operations. For example a counter that you want to increment. In async its very rare you have a shared global state, but even if you do, reading and modifying the state is naturally right next to each other, not on opposite sides of the await.
- jerf 10y ago
  "On async, context switching onlykhappens on an "await", which means race conditions won't happen. Race conditions typically happen when you have a shared global state and you read and write to it in two operations." That may be when the "typically" happen, but you can still get race conditions where you have two tasks that can run next, and you accidentally write an assumption about which will happen into your code when there is no such assumption in the scheduler. You certainly will get fewer of these with async/await than with pure event-handling-style code, because async/await carries more information about proper ordering of code, but yes, you can still get things that are correctly described as "race conditions".
  - rudolf0 10y ago
    >That may be when the "typically" happen, but you can still get race conditions where you have two tasks that can run next, and you accidentally write an assumption about which will happen into your code when there is no such assumption in the scheduler. What are some async coding styles where this is prevented? I feel like making incorrect order-of-execution/arrival time assumptions could be an issue for any kind of async code.
    - jerf 10y ago
      I'm not sure 100% prevention is an option. Working in a language with very strong mutation control, either via immutability in Haskell or something very controlling like Rust, probably eliminates most of them, but I'm pretty sure in both cases it would still be possible to write a straight-up logic error where the programmer assumed a guarantee that did not exist. Still, I recommend that approach personally. The best of all worlds is to write threaded code in languages that structurally eliminate the majority of issues threads have. The downside is that the "languages that structurally tame threads" is a pretty short list, even now; Rust, Haskell, Erlang/Elixir, see the probably-inevitable replies for a couple of others. You can widen the field substantially by working with libraries in languages that encourage that style of programming, but lacking the language support means you're back to exercising a lot of programmer discipline to ensure you stay in the acceptable subset of the language. This is a continuum based on the support available in the languages and the ability to get libraries, but this gets you to the Go language, Java/Scala's Akka, and some stuff like that. I'm still not really willing to put anything Python has into even the latter list, though; the dynamic languages are just so easy to blow your foot off. (I still like "green threads" personally, because I write network servers where tens of thousands of simultaneous connections is a realistic concern and the price is a small one to pay. I tend to agree with the Rust community that people are generally grotesquely overestimating the run-time costs of threads nowadays. If you're in a language like Rust, and you're working in a program where you can be pretty confident you're never going to spawn 100,000 threads or face unbounded thread creation for some reason, go for it. Threads are not magically bad; they're bad for specific reasons, and if you address those specific reasons, they cease being ravening Elder Gods who eat your soul, and become just another professional-grade tool... yeah, it has some sharp points on it, but it does the job really well.)
hueving 10y ago
> as soon as you add a new yield point, you invalidate your safety assumptions. While true, locks aren't free from this problem. They have the inverse. If someone adds code that accesses a data structure that should be protected by a lock and they forget to add the lock, you also lose all of your safety assumptions.

vomjom 10y ago

It's not clear what you're suggesting as an alternative. My understanding is that you're suggesting thread-per-request, which has many known flaws. There are three approaches to serving requests: 1. Thread-per-request. This is a simple model. You have a fixed-size thread pool of size N, and once you hit that limit, you can't serve anymore requests. Thread-per-request has several sources of overhead, which is why people recommend against it: thread limits, per-thread stack memory usage, and context switching. 2. Coroutine style handling with cooperative scheduling at synchronization points (locks, I/O). This is how Go handles requests. 3. Asynchronous request handling. You still have a fixed-size thread pool handling requests, but you no longer limit the number of simultaneous requests with the size of that thread pool. There are several different styles of async request handling: callbacks, async/await, and futures. #2 and #3 are more common these days because they don't suffer from the many drawbacks of the thread-per-request model, although both suffer from some understandability issues.

quotemstr 10y ago
Those options aren't as distinct as you might imagine. Would calling it fiber-per-request make you happy? (By the way: most of the time, a plain-old-boring thread-per-request is just fine, because most of the time, you're not writing high-scale software. If you have at most two dozen concurrent tasks, you're wasting your time worrying about the overhead of plain old pthread_t.) I'm using a much more expansive definition of "thread" than you are. Sure, in the right situation, maybe M:N threading, or full green threads, or whatever is the right implementation strategy. There's no reason that green threading has to involve the use of explicit "async" and "await" keywords, and it's these keywords that I consider silly.
- vomjom 10y ago
  (I agree that thread-per-request works just fine in the majority of cases, but it's still worthwhile to write about the cases where it doesn't work.) Responding to your original post: you argue that async/await intends to solve the problem of data races. That's not why people use it, nor does it tackle that problem at all (you still need locks around shared data). It only tries to solve the issue of highly-concurrent servers, where requests are bound by some resource that a request-handling threads have to wait for the result of (typically I/O). Coroutines/fibers are not an alternative to async servers, because they need primitives that are either baked into the language or the OS itself to work well.
  - cderwin 10y ago
    Please correct me if I'm wrong, but doesn't asyncio in the form of async/await (or any other ways to explicitly denote context switches) solve the problem of data races in that per-thread data structures can be operated on atomically by different coroutines? My understanding is that unless data structures are shared with another thread, you don't usually need locks for shared data.
    - omribahumi 10y ago
      I think that the biggest argument against it is code changes. Think about a code change that adds an additional yield point without proper locking. Has any language tackled this with lazy locking? i.e. lock only on yield. Maybe this could even be done in compile time

kmike84 10y ago

In his talk about threads (https://www.youtube.com/watch?v=Bv25Dwe84g0 https://www.youtube.com/watch?v=Bv25Dwe84g0) Raymond Hettinger made a point: when you have a codebase with many locks the result of composition is often a sequential program.

Arnt 10y ago

I think the general idea is that if you make your yield points explicit, you see them yourself, and if you generally try for 100% unit test coverage, you end up nudging yourself towards fewer yield points, a simpler state machine, and better karma overall. The complexity you see affects yourself more than the complexity you don't. I rather agree. FWIW my friend Abhijit Menon-Sen wrote a blog post on the matter last year, about some code with excellent test coverage and explicit yield points: http://toroid.org/callback-heaven http://toroid.org/callback-heaven

lmm 10y ago

> Look: in traditional multi-threaded programs, we protect shared data using locks. If you avoid explicit locks and instead rely on complete knowledge of all yield points (i.e., all possible execution orders) to ensure that data races do not happen, then you've just created a ticking time-bomb: as soon as you add a new yield point, you invalidate your safety assumptions. > Traditional lock-based preemptive multi-threaded code isn't susceptible to this problem: it already embeds maximally pessimistic assumptions about execution order, so adding a new preemption point cannot hurt anything. You get an equal and opposite problem: whenever you add one more lock, you invalidate your liveness assumptions. > The article claims that yield-based programming is easier because the fewer the total number of yield points, the less mental state a programmer needs to maintain. I don't think this argument is correct: in lock-based programming, we need to keep _zero_ preemption points in mind, because we assume every instruction is a yield point. Instead of thinking about NY program interleavings, we think about how many locks we hold. I bet we have fewer locks than you have yields. I'll take that bet. You really don't have to yield very often - only when making a network request, and perhaps not even for that in the case of a fast local network. Whereas you have to lock every piece of state that you have.

IgorPartola 10y ago
Every piece of shared state. In the case of something like a web server, you only need one lock: when the connection is being accepted and handed off to a worker thread. How many reads and writes does your web server perform?
marcosdumay 10y ago
> Whereas you have to lock every piece of state that you have. You need to lock every piece of shared state you have. Where "shared" means stuff that many threads must communicate among themselves. One tends to keep the number of that kind of state low, really low. When zero is not possible, the most common number by a wide margin is one¹. If you have more than 1, they are normally completely independent pieces of state that will not be used at the same time. If you have more than 1, and they are not independent, the code is either the result of at least one PHD thesis, or it does not work (or, often, both). I bet you do network requests more than once on your code. 1 - The size of the shared state does not matter, so it's often one really big state.

nhumrich 10y ago

Your saying to say that async will still have race conditions, but I disagree with that premise. On async, context switching onlykhappens on an "await", which means race conditions won't happen. Race conditions typically happen when you have a shared global state and you read and write to it in two operations. For example a counter that you want to increment. In async its very rare you have a shared global state, but even if you do, reading and modifying the state is naturally right next to each other, not on opposite sides of the await.

jerf 10y ago
"On async, context switching onlykhappens on an "await", which means race conditions won't happen. Race conditions typically happen when you have a shared global state and you read and write to it in two operations." That may be when the "typically" happen, but you can still get race conditions where you have two tasks that can run next, and you accidentally write an assumption about which will happen into your code when there is no such assumption in the scheduler. You certainly will get fewer of these with async/await than with pure event-handling-style code, because async/await carries more information about proper ordering of code, but yes, you can still get things that are correctly described as "race conditions".
- rudolf0 10y ago
  >That may be when the "typically" happen, but you can still get race conditions where you have two tasks that can run next, and you accidentally write an assumption about which will happen into your code when there is no such assumption in the scheduler. What are some async coding styles where this is prevented? I feel like making incorrect order-of-execution/arrival time assumptions could be an issue for any kind of async code.
  - jerf 10y ago
    I'm not sure 100% prevention is an option. Working in a language with very strong mutation control, either via immutability in Haskell or something very controlling like Rust, probably eliminates most of them, but I'm pretty sure in both cases it would still be possible to write a straight-up logic error where the programmer assumed a guarantee that did not exist. Still, I recommend that approach personally. The best of all worlds is to write threaded code in languages that structurally eliminate the majority of issues threads have. The downside is that the "languages that structurally tame threads" is a pretty short list, even now; Rust, Haskell, Erlang/Elixir, see the probably-inevitable replies for a couple of others. You can widen the field substantially by working with libraries in languages that encourage that style of programming, but lacking the language support means you're back to exercising a lot of programmer discipline to ensure you stay in the acceptable subset of the language. This is a continuum based on the support available in the languages and the ability to get libraries, but this gets you to the Go language, Java/Scala's Akka, and some stuff like that. I'm still not really willing to put anything Python has into even the latter list, though; the dynamic languages are just so easy to blow your foot off. (I still like "green threads" personally, because I write network servers where tens of thousands of simultaneous connections is a realistic concern and the price is a small one to pay. I tend to agree with the Rust community that people are generally grotesquely overestimating the run-time costs of threads nowadays. If you're in a language like Rust, and you're working in a program where you can be pretty confident you're never going to spawn 100,000 threads or face unbounded thread creation for some reason, go for it. Threads are not magically bad; they're bad for specific reasons, and if you address those specific reasons, they cease being ravening Elder Gods who eat your soul, and become just another professional-grade tool... yeah, it has some sharp points on it, but it does the job really well.)

hueving 10y ago

> as soon as you add a new yield point, you invalidate your safety assumptions. While true, locks aren't free from this problem. They have the inverse. If someone adds code that accesses a data structure that should be protected by a lock and they forget to add the lock, you also lose all of your safety assumptions.