3 ms·
> internal consistency as defined here seems to mean how "multiple reads" can lock into the same storage state Sort of. When you make a single read of a single
by jamii 5y ago
> internal consistency as defined here seems to mean how "multiple reads" can lock into the same storage state
Sort of. When you make a single read of a single key at the end of a streaming topology, that's equivalent to reading a all the inputs and doing the computation yourself. Internal consistency means that every path through the graph back to the the inputs should be reading the same set of inputs. Things go wrong when one path has processed more inputs than another, or when a single input produces multiple internal updates and those updates aren't processed atomically.
> isn't it possible for subsequent reads to lock into the maximum transaction id of the first read
Your example would still allow credits to have read more inputs than debits, but it's on the right track.
In differential dataflow aggregates wait until they see a watermark before emitting output, so the sum in `balance` will wait until the watermark for eg time 7 before emitting the sum for time 7. That allows the output from the join to settle.
Of course doing that efficiently is non-trivial...