6 ms·

I am not sure of specifics on features, but I think the fundamental difference is that Arroyo is a stream processing engine i.e., it doesn’t have a database, wh

• necubi 3y ago
  Yes, that's mostly true. Arroyo is a stream processor like Flink. In Arroyo, you pre-register the queries you are interested in, and they will be compiled into streaming dataflow jobs that continuously execute as events come in. This means that we're not storing all of the raw events that come in for later querying. However, like a database we do have a serving layer (currently only in our cloud version due to its reliance on our distributed state backend: https://doc.arroyo.dev/connectors/state https://doc.arroyo.dev/connectors/state) so it is possible to query the results directly from Arroyo as well. Generally you would want to use something like Arroyo when your data is too high volume to reasonably store it all in a DB like Clickhouse, or your queries are too expensive to perform on every query, as Arroyo incrementally computes the results of the query as events come in. There's also opportunities to use these systems together: Arroyo can pre-aggregate the high volume raw data, and then it can be inserted into a Clickhouse-based system for final processing along different dimensions.