For the data processing tasks my go-to tool is Manifold[1]. Depending on how the application is structured, you can keep state in atoms, the system map when usi

For the data processing tasks my go-to tool is Manifold[1]. Depending on how the application is structured, you can keep state in atoms, the system map when using Component[2] or vars with Mount[3]. Manifold streams asynchronously join the state transformations together while allowing control over backpressure and threading. On the data analysis side things aren't rosy on the JVM. I'd delegate that part to something which actually has all the tools built-in, e.g. Julia/Python+Numpy/R. Apache Arrow[4] is a nice project to facilitate the dataframe interop. [1] https://github.com/ztellman/manifold https://github.com/ztellman/manifold [2] https://github.com/stuartsierra/component https://github.com/stuartsierra/component [3] https://github.com/tolitius/mount https://github.com/tolitius/mount [4] https://arrow.apache.org/ https://arrow.apache.org/