Sort of. Let's say you're counting the number of unique users visiting your site in a 5 minute fixed window. This is pretty simple: you have a set, and each tim

Sort of. Let's say you're counting the number of unique users visiting your site in a 5 minute fixed window. This is pretty simple: you have a set, and each time a request comes in you add it to the set. At the end of the 5 minute window you output the set cardinality to your downstream consumer and throw out the raw data. But what if this is a mobile app, and requests may be delayed by minutes or even hours due to gaps in cellular connectivity? In that case, we need a strategy to handle "late arriving" data. Flink has several approaches for this, but they're all based around a "high-watermark." This is basically an estimate at time T that all data from time T-D has arrived. Once the watermark has passed the end of our window we consider it closed and can compute the final value for it. (How you compute the watermark is up to you; typically you use a fixed value but this can be made more accurate if you have out-of-band information). In addition to the default behavior (where late-arriving data is dropped) you can customize this by specifying a trigger that runs when late-arriving date comes, which can be used to e.g., update an external datastore. However at that point the reconciliation is outside of the scope of Flink. There are various other ways to organize this within flink, i.e., you can keep the windows open indefinitely, update the internal state of your flink job, and serve directly from the flink state, or you can use a periodic trigger to periodically update your downstream from that state. Obviously this will require a larger state size (in disk or in memory depending on your state backend) since you won't be able to close out the window and have to keep all of the data around. Much more about all this in the Flink docs (https://ci.apache.org/projects/flink/flink-docs-release-1.11/concepts/timely-stream-processing.html https://ci.apache.org/projects/flink/flink-docs-release-1.11...). Also a lot of the theory behind this comes from the Google Dataflow paper (https://research.google/pubs/pub43864/ https://research.google/pubs/pub43864/) which is also just a great read.