3 ms·

Hey, thanks for the comment. A few clarifications you asked about: 3. One of the main use cases for using external databases and stores is that they can be mu

Hey, thanks for the comment. A few clarifications you asked about: 3. One of the main use cases for using external databases and stores is that they can be multi tenant. Bob comes up with a really cool market aggregation, Susan and Ralph want to use that aggregation in their models. Currently, is there a way for Susan and Ralph to just hook into that aggregation in memory and allow them to use that as a feature? 4. The main use case is that your streaming framework makes a bunch of cool aggregates, but if you have a customer who needs the question “Should I let this person do this thing?”, you want to answer that synchronously. The alternative is to load your state into an external store and setup a web server that just does a lookup on that store, instead of doing it in your streaming system.

spooneybarger 9y ago
3. currently, there's no adhoc way to query in-memory data. wallaroo while having a couple database like features isn't a database. you'd need to export that aggregation or the results thereof via a sink into some other system for adhoc querying. we've had discussions about providing adhoc querying but it isn't on the immediate radar. 4. we've worked with folks who want to do what you are describing but the "answer channel" was different than the "question channel". so the question came in over a source (in their case TCP) and the answer would leave over a sink (also TCP). in that case it was something like: client -> persistent web socket -> server -> streaming system -> server -> persistent web socket -> client having a combined source/sink is something that we've discussed but we have higher priorities at the moment. if we were to start working with someone for whom it was a priority, we'd move it up in priority.