15 ms·
What I'm thinking of is flushing the table to a secondary storage so that other services can query that data. I think Storm/Flink have the concept of a "tick t
by nemothekid 8y ago
What I'm thinking of is flushing the table to a secondary storage so that other services can query that data.
I think Storm/Flink have the concept of a "tick tuple", a message that comes every n-seconds to tell the worker to flush the table to some other store. I've been looking over the project, and I'm not sure how I would do this in Faust yet, as far as I understand the "Table" is partitioned, so you'd have to send a tick to every worker.
Very interesting project!
- asksol 8y agoFaust can serve the data over HTTP/websockets and other transports, so you can query it directly!
- vineetgoel1992 8y agoIn addition to that you can have an arbitrary method run at an interval using the app.timer decorator. You can use this to flush every n seconds. You could also use stream.take to read batches of messages (or wait t seconds, whichever happens first) and write to an external source.