3 ms·
We're currently moving our Druid/Kafka/Spark setup to BigQuery because of how simple SQL is.
by KingOfCoders 5y ago
We're currently moving our Druid/Kafka/Spark setup to BigQuery because of how simple SQL is.
- hashhar 5y agoYou might find https://trino.io/ https://trino.io/ interesting. It allows you to bolt on a MPP SQL execution engine on top of any data source including pre-built connectors for Druid and Kafka. It's all ANSI SQL and the best part is you can combine data from heterogenous sources. e.g. You can join data between a topic in Kafka and a table in Druid or even between Kafka, S3 and your RDBMS. Disclaimer: I'm a maintainer of the project.
- dominotw 5y agoMy employer is going to the other way from snowflake -> druid to reduce high response latencies/ improve freshness. I am assuming your use cases can tolerate increased latency?
- KingOfCoders 5y agoYes they can, we will move low latency to a smaller, different pipeline.