You can also inject pull queries at some intermediate location into the dataflow, either in the form of a Dirac-delta for a instant-in-time query, or persisting

You can also inject pull queries at some intermediate location into the dataflow, either in the form of a Dirac-delta for a instant-in-time query, or persisting to receive push updates for the query results. [0] makes it fairly easy to handle, via, say, inserting the persistent queries into a native table of it (which don't typically get persisted, though that could have changed in the past few months), and the keep-alive messages from the viewer bump the expire-at field in the query entry in the table. [1] is an example of how exactly the internal dataflow of such "demand-driven push" works, though it omits the expiry handling you'd want, and doesn't detail the interaction with the outside needed for live feeds. What this doesn't emphasize is that due to the timestamping and consistent nature of the underlying dataflow engine, you can see when exactly the new query is in your output stream, and also e.g. bundle multiple queries together into one atomic transaction, to not get any tearing-style output/display artifacts. The underlying Rust framework Differential Dataflow[2] is even more powerful, but also far less easy to use, due to the lack of a query optimizer. It arguably makes up by already supporting recursive/iterative computation and multi-temporal timestamps. [0]: https://materialize.com/a-simple-and-efficient-real-time-application-powered-by-materializes-tail-command/ https://materialize.com/a-simple-and-efficient-real-time-app... [1]: https://materialize.com/lateral-joins-and-demand-driven-queries/ https://materialize.com/lateral-joins-and-demand-driven-quer... [2]: https://github.com/TimelyDataflow/differential-dataflow https://github.com/TimelyDataflow/differential-dataflow