3 ms·
This is a different use case - DuckDB is large queries over your entire dataset; Arroyo is continuous queries executing over a live data stream. So if you have
by jakewins 3y ago
This is a different use case - DuckDB is large queries over your entire dataset; Arroyo is continuous queries executing over a live data stream.
So if you have a stream of sensor data from a bunch of IoT devices, you might use Arroyo to run “live” queries over the flowing data, and you might use DuckDB to do analytical queries over archive data
Although they are both hammers to whack nails with I guess, so a lot of use cases likely work well to solve with both systems
- mgaunard 3y agoDuckDB processes data lazily, and can easily be integrated with custom synthetic relations. I guess it's still not really designed to work well with infinite relations, but then again neither is SQL really.
- jakewins 3y agoMaybe I’ve not used DuckDB enough but: How would you set it up to consume, say, a Kafka stream, apply some transformations or windowed aggregates and continuously output the result? In my experience DuckDB is a batch tool, vs this thing which is a continuous stream tool
- mgaunard 3y agoYou would implement the relation type that maps to your data source.