3 ms·
I've been thinking a bit since the last article from the author on Internal Consistency [1] on HN [2] on how to stress test a framework, or an application writt
by BenoitP 5y ago
I've been thinking a bit since the last article from the author on Internal Consistency [1] on HN [2] on how to stress test a framework, or an application written on it.
The problem is: it is hard to know in advance what kind of pressures your production system will be submitted to.
I think the right approach would be Jepsen-like: generate inputs and delay them by inconveniently crafted amounts. Maybe take a look at the execution graph to infer what to delay (for example Rows that will be joined further down the graph). One could implement dedicated operators for DataStreams; and for SQL some bytecode could be injected in Flink's Operator.processElement to add delays.
[1] https://scattered-thoughts.net/writing/internal-consistency-in-streaming-systems/ https://scattered-thoughts.net/writing/internal-consistency-...
[2] https://news.ycombinator.com/item?id=26851803 https://news.ycombinator.com/item?id=26851803