5 ms·

Good job at highlighting async as being cooperative multi-tasking. Those of us old enough will remember the fanfare when both Mac and Windows moved from cooper

Good job at highlighting async as being cooperative multi-tasking. Those of us old enough will remember the fanfare when both Mac and Windows moved from cooperative to pre-emptive multi-tasking models. To adapt a well known phrase, every non-trivial cooperative application includes a half-baked, buggy implementation of a pre-emptive scheduler. We know that pre-emptive scheduling of isolated processes is a good abstraction. It's non-trivial to write a scheduler; certainly not something that every app developer should need to deal with. I've said before and I'll say again: this kind of async is a design flaw. It's a retrograde step that brings all the cognitive overload of cooperative multitasking back. If the OS doesn't offer enough processes for some reason (e.g. too heavyweight) the answer should not be to throw our hands up and regress. The better answer is to work out how to provide more processes. It's absolutely doable: look at Erlang. Of course things can improve still further; e.g. Dataflow variables [0] provide an effective abstraction for processing pipelines. [0]: https://en.wikipedia.org/wiki/Oz_%28programming_language%29 https://en.wikipedia.org/wiki/Oz_%28programming_language%29

sametmax 8y ago
Cooperative multi-tasking if terrible is you don't have control over what the others are doing, but in your program you generally do. It also have the benefit of making concurrent access much easier to reason about without having to resort to message passing or immutability, yet locks are rarely necessary. As usual, it's a trade off.
- greglindahl 8y ago
  I've learned (the hard way) that any in any project with more than one programmer, sometimes you don't have control over what the others are doing. And on a bad day, I can do "the bad thing" to myself. One way to help mitigate this is good tooling. A warning that a piece of async computation took longer than expected can be very useful if you expect everything to finish in a few tens of milliseconds.
maayank 8y ago
Can't agree more. As someone who've done lots of work on both preemptive- and cooperative- multitasking codebases I'm perplexed by how many frameworks dive head-first into the latter.
- lotia 8y ago
  Interesting observation. Any theories on why? As someone who has consumed but not written much by way of schedulers I’d be keen to hear thoughts.
  - maayank 8y ago
    (I assume you're asking why frameworks delve headfirst into coop, please correct me if I'm wrong) Cooperative multitasking has much of the semantics (and thus semblance) of preemptive multitasking while also having the allure of not needing the harder (either with respect to understanding or actual usage) semantics of locking, the mental accounting related to thread safe algorithms, etc. In other words, you get the semantics of concurrency (e.g. tasks and working queues) without the overhead of synchronizing. That sounds very alluring, but in practice I would like to not delve into specialized semantics AT ALL unless I'm expecting some speed/scalability gains, where the bulk of it comes from actual parallel execution, either in the form of different cores or different nodes. And then you need in any case to use the same semantics concurrent (i.e. cooperative) frameworks try to avoid, e.g. in the form of synchronizing between processes but with much less tooling because this is a second-tier scenario in most cooperative multitasking frameworks I've seen. In other words, you don't get the speedups that are worth the mental effort of using the semantics in the first place. The only exception I've seen is a code base properly utilizing their I/O "breaks" for cooperation and indeed I see cooperative multitasking frameworks often touting I/O intensive applications as an ideal candidate. But, please note that: 1. Programmers usually overestimate how much I/O (or any resource usage) blocks. The codebase I was referring to was an actual filesystem. 2. Behind toy programs you still need to do a lot of thread-safe type accounting. You're not susceptible to cache-coherency issues, but that's about it. Phew, didn't think I have so much to say about it.
naasking 8y ago
> I've said before and I'll say again: this kind of async is a design flaw. It's a retrograde step that brings all the cognitive overload of cooperative multitasking back. You seem to be assuming that preemption is always needed or wanted. This is not always the case. The lack of preemption actually simplifies a number of factors that simply aren't relevant in async scenarios in a single program. async/await patterns are actually the perfect solution for capturing the minimal context necessary for resumption after the async operation completes.
0x445442 8y ago
I'm currently working on a project that's implementing this whole pattern using AWS. 1.) Requests are written to a DynamoDB table 2.) a Lambda subscribes to the Dynamo event stream 3.) each event from the stream causes the Lambda to make an async call to an external system 4.) A message (with a delay) is sent to SQS from the Lambda that the async call has been made 5.) Another Lambda reads from the SQS queue and updates the original DynamoDB record completing the loop. The delay in the queue simulates a polling interval to the external system and when a successful response is returned from the external system the Dynamo record is updated with a status telling the Lambda subscribing to the event stream that "the loop" is done. This setup has to run at fairly significant peak loads and we're relying on Amazon to provision resources reliably enough to pull this off. None of us know if this will work but in terms of abstracting away the complexities of async/concurrent processing it's a novel approach.
- greglindahl 8y ago
  It's similar to how you use Linda (1986), which wasn't novel when it was invented, either.
- Matthias247 8y ago
  The challenge with that kind of system is flow control. Essentially everything is coupled by queues without a feedback loop. And once the system gets under load, those queues might run fuller and fuller until the whole system fails. It’s actually similar than an in-program model where you just asynchronously queue void functions in threadpools without knowing how many actually run. The fact that you are only running on top of managed services and that AWS can scale really well might mitigate the problem. And if you have to process all events and can not fail early or move backpressure on the initial caller there might not really be a better solution anyway. But you should be aware about it.
greglindahl 8y ago
I've been doing cooperative multitasking using dataflow for a couple of decades; your terms are fuzzy enough that you're criticizing the problems of particular implementations and not fundamental flaws in the concepts. As an example of cooperative multitasking with dataflow, consider single-process nginx. Events are delivered using epoll; nginx has an event loop that does work on the next task that has data ready (or a timer event.) This is called the Reactor Pattern in the article.

sametmax 8y ago

Cooperative multi-tasking if terrible is you don't have control over what the others are doing, but in your program you generally do. It also have the benefit of making concurrent access much easier to reason about without having to resort to message passing or immutability, yet locks are rarely necessary. As usual, it's a trade off.

greglindahl 8y ago
I've learned (the hard way) that any in any project with more than one programmer, sometimes you don't have control over what the others are doing. And on a bad day, I can do "the bad thing" to myself. One way to help mitigate this is good tooling. A warning that a piece of async computation took longer than expected can be very useful if you expect everything to finish in a few tens of milliseconds.

maayank 8y ago

Can't agree more. As someone who've done lots of work on both preemptive- and cooperative- multitasking codebases I'm perplexed by how many frameworks dive head-first into the latter.

lotia 8y ago
Interesting observation. Any theories on why? As someone who has consumed but not written much by way of schedulers I’d be keen to hear thoughts.
- maayank 8y ago
  (I assume you're asking why frameworks delve headfirst into coop, please correct me if I'm wrong) Cooperative multitasking has much of the semantics (and thus semblance) of preemptive multitasking while also having the allure of not needing the harder (either with respect to understanding or actual usage) semantics of locking, the mental accounting related to thread safe algorithms, etc. In other words, you get the semantics of concurrency (e.g. tasks and working queues) without the overhead of synchronizing. That sounds very alluring, but in practice I would like to not delve into specialized semantics AT ALL unless I'm expecting some speed/scalability gains, where the bulk of it comes from actual parallel execution, either in the form of different cores or different nodes. And then you need in any case to use the same semantics concurrent (i.e. cooperative) frameworks try to avoid, e.g. in the form of synchronizing between processes but with much less tooling because this is a second-tier scenario in most cooperative multitasking frameworks I've seen. In other words, you don't get the speedups that are worth the mental effort of using the semantics in the first place. The only exception I've seen is a code base properly utilizing their I/O "breaks" for cooperation and indeed I see cooperative multitasking frameworks often touting I/O intensive applications as an ideal candidate. But, please note that: 1. Programmers usually overestimate how much I/O (or any resource usage) blocks. The codebase I was referring to was an actual filesystem. 2. Behind toy programs you still need to do a lot of thread-safe type accounting. You're not susceptible to cache-coherency issues, but that's about it. Phew, didn't think I have so much to say about it.

naasking 8y ago

> I've said before and I'll say again: this kind of async is a design flaw. It's a retrograde step that brings all the cognitive overload of cooperative multitasking back. You seem to be assuming that preemption is always needed or wanted. This is not always the case. The lack of preemption actually simplifies a number of factors that simply aren't relevant in async scenarios in a single program. async/await patterns are actually the perfect solution for capturing the minimal context necessary for resumption after the async operation completes.

0x445442 8y ago

I'm currently working on a project that's implementing this whole pattern using AWS. 1.) Requests are written to a DynamoDB table 2.) a Lambda subscribes to the Dynamo event stream 3.) each event from the stream causes the Lambda to make an async call to an external system 4.) A message (with a delay) is sent to SQS from the Lambda that the async call has been made 5.) Another Lambda reads from the SQS queue and updates the original DynamoDB record completing the loop. The delay in the queue simulates a polling interval to the external system and when a successful response is returned from the external system the Dynamo record is updated with a status telling the Lambda subscribing to the event stream that "the loop" is done. This setup has to run at fairly significant peak loads and we're relying on Amazon to provision resources reliably enough to pull this off. None of us know if this will work but in terms of abstracting away the complexities of async/concurrent processing it's a novel approach.

greglindahl 8y ago
It's similar to how you use Linda (1986), which wasn't novel when it was invented, either.
Matthias247 8y ago
The challenge with that kind of system is flow control. Essentially everything is coupled by queues without a feedback loop. And once the system gets under load, those queues might run fuller and fuller until the whole system fails. It’s actually similar than an in-program model where you just asynchronously queue void functions in threadpools without knowing how many actually run. The fact that you are only running on top of managed services and that AWS can scale really well might mitigate the problem. And if you have to process all events and can not fail early or move backpressure on the initial caller there might not really be a better solution anyway. But you should be aware about it.

greglindahl 8y ago

I've been doing cooperative multitasking using dataflow for a couple of decades; your terms are fuzzy enough that you're criticizing the problems of particular implementations and not fundamental flaws in the concepts. As an example of cooperative multitasking with dataflow, consider single-process nginx. Events are delivered using epoll; nginx has an event loop that does work on the next task that has data ready (or a timer event.) This is called the Reactor Pattern in the article.