3 ms·
I was moreso referring to the pieces of queries that Calcite may have to execute itself, often because they are not supported by the underlying data source. Tha
by michaelmior 5y ago
I was moreso referring to the pieces of queries that Calcite may have to execute itself, often because they are not supported by the underlying data source. That piece of Calcite hasn't been very highly optimized. There's definitely ongoing work in that area though.
For example, I have a former student who's been working on an adapter to Apache Arrow. Beyond having a fast execution engine, this should also eventually enable Calcite to work pretty seamlessly with other systems using Arrow without serialization overhead.
- gavinray 5y agoIs this student in contact with Jacques Nadeau? He's one of the only people I know that's a committer/PMC of both Arrow and Calcite, super nice and brilliant guy. Maybe he'd have good feedback or ideas around this? https://twitter.com/intjesus https://twitter.com/intjesus
- michaelmior 5y agoIndirectly. I know Jacques. I'm also on the Calcite PMC :) Thanks for the pointer though!
- gavinray 5y agoIf you've responded to any of my emails, no thank YOU!