4 ms·
Yeah, you definitely need backpressure, and doing that well is had. There's a great blog post about how Flink manages this [0] and it involves a complex credit-
by necubi 3y ago
Yeah, you definitely need backpressure, and doing that well is had. There's a great blog post about how Flink manages this [0] and it involves a complex credit-based flow control system.
But in practice the performance advantages of allowing operators to do work as they have capacity (rather than being potentially starved by the capacity of their downstreams) outweighs this.
[0] https://flink.apache.org/2019/06/05/a-deep-dive-into-flinks-network-stack/ https://flink.apache.org/2019/06/05/a-deep-dive-into-flinks-...