4 ms·
Precisely. Conduits basically do everything on demand. For parallel processing the backpressure becomes a bit more complicated, in particular if you end with a
by yoricksijsling 4y ago
Precisely. Conduits basically do everything on demand.
For parallel processing the backpressure becomes a bit more complicated, in particular if you end with a sequential consumer. If you're just doing your parallel tasks when a sequential consumer demands it, you'll find yourself still doing one task at a time. The straightforward solution is to work ahead a bit, spawning just enough parallel work to fill a small buffer that the consumer can read from. You get backpressure this way, but may have done a little bit too much if the consumer doesn't want the rest of the data. We'll show this in the third blog post of the series.
- still_grokking 4y ago> We'll show this in the third blog post of the series. Thanks for the pointer! Will have a look.