3 ms·
Hi, author here, While yes, running a coroutine in a new task is in the standard library, that is only a component of what this library does, which is: - give
by michalc 6y ago
Hi, author here,
While yes, running a coroutine in a new task is in the standard library, that is only a component of what this library does, which is:
- given an async iterable,
- iterating over it in a new task, into a buffer,
- and then returning an iterable that yields values from this buffer as it's filled.
I don't _think_ this logic is in the standard library, at least not at https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task https://docs.python.org/3/library/asyncio-task.html#asyncio.... from what I can see.
- scaramanga 6y agoYeah, this actually looks pretty cool, it's a nice way to abstract a common pattern which is otherwise quite tedious, namely, that of spawning a bunch of tasks and have them feed each other work via asyncio queues. Have you thought about making it work as a decorator?
- michalc 6y agoIndirectly: the initial version was a single function, so could have been used in a decorator easily enough. The issue is what happens on exceptions to avoid tasks hanging around forever. Exceptions propagate “forward” in the pipeline fairly automatically. However, getting the ones “behind” the exception to notice: that was tricky. The best I came up with was the bit of state wrapped up in buffered_pipeline that notices the exceptions and cancels the tasks.