4 ms·
Sad to see this happen as I really like the idea! If you're interested in Drill, check out OctoSQL[0]. It shares the same vision of querying multiple datasourc
by cube2222 6y ago
Sad to see this happen as I really like the idea!
If you're interested in Drill, check out OctoSQL[0]. It shares the same vision of querying multiple datasources using pure SQL, and pushing down as much operations as possible to the underlying datasource.
Moreover, there's been a huge rewrite under way this past year, ready to use on the master branch, yet unreleased however (will be available soon).
It adds Kafka and Parquet support and most importantly first-class unbounded stream support. Including temporal SQL extensions for working with event time metadata (instead of system time) so you can use stuff like live updated time window aggregations on incoming kafka streams. It also now uses on-disk badger storage as the primary way to store its state, so you can do Group Bys / Joins with lots of keys, and restarts of OctoSQL won't alter the final result (exactly-once semantics).
Make sure to check it out, it's also very simple to get going locally!
Disclosure: I'm one of the main contributors.
[0]:https://github.com/cube2222/octosql https://github.com/cube2222/octosql
- bsg75 6y agoDoes OctoSQL support reading columnar compressed formats (ex. Parquet) from distributed storage (ex. S3) ?
- willvarfar 6y agoHow does it compare to prestosql?
- cube2222 6y agoMuch less mature. There's no distributed execution yet. Easy to get going locally. I'm not sure but I think Presto is in-memory? We're optimizing for SSD disks to easily support big states, and achieve durability this way. SSD disks are still plenty fast. Definitely better streaming support. We're very much concentrating on good streaming ergonomics. OctoSQL isn't a batch execution engine at its heart, it's a streaming one.