> The problem with Kappa architecture that other commenters here are highlighting is that this results in holding a large m in memory at all times (fragile, exp

> The problem with Kappa architecture that other commenters here are highlighting is that this results in holding a large m in memory at all times (fragile, expensive), and handling out of order events by re-doing the live operation on them - if you do (m, d_505) and then d_504 drops in after, you have to do ((m, d_504), d_505). This is true for time-sensitive workloads, and most business applications do not have such a requirement and typically have some leeway to tolerate the untimliness of the event arrival. Provided the event has an epoch associated with it that can unambiguously reference the point in space and time when the event had occurred, the time ordered sequence of events can be reconstitued by using tumbling (or another window type) windows (e.g. Kafka and similar) or database window functions. The now time ordered sequence of events can then be flown into a new topic (Kafka) or a DB table/document collection and ingested out of that one in the ordered sequence. RE: batching and real-time. The only real difference is that real time data is constant vs burst, i.e. a trick flow of data vs a scheduled at regular intervals surge. But a batch can be de-batched and unwound into a series of isolated events at the ingestion point – to become more «real-time» and to not require to be processed as an atomic unit. Therefore, if the untimeliness can be tolerated to a certain degree, there is no real need to treat real-time and batched up data separately.