This blog post is the first mention I'd seen of Gandiva: https://github.com/dremio/gandiva https://github.com/dremio/gandiva As I began reading about that, I

This blog post is the first mention I'd seen of Gandiva: https://github.com/dremio/gandiva https://github.com/dremio/gandiva As I began reading about that, I thought it sounded a lot like Weld, so I was happy to see it listed in the prior art: https://www.weld.rs https://www.weld.rs These projects present a compiler IR that can be emitted by an analytics framework. For example, Spark and pandas currently run functions directly to perform filters and aggregations. These can be fast for simple user cases, but complex user code might result in a lot of branching and temporary values. So instead, the libraries could emit a mid-level IR that is then JITed via LLVM. That's exactly what Weld is for and it looks like Gandiva is similar (on top of Arrow data). I've seen a lot of interest in LLVM-based array programming lately, including TensorFlow XLA and Intel's nGraph. So I'm glad to see these techniques applied to tabular data as well.