3 ms·
The section titled "An Overview of Velox" gets into the meat of it - you give it some data and an optimised plan of the operators you want applied (expression,
by jsty 4y ago
The section titled "An Overview of Velox" gets into the meat of it - you give it some data and an optimised plan of the operators you want applied (expression, evaluation, aggregation, etc.) - Velox then executes that plan as efficiently as possible given the available compute resources.
That way multiple top-level systems like analytics databases, dataframe implementations, etc. can all share the same underlying execution engine.
- picardo 4y agoSo it's like an operating system for cloud workloads?
- aseipp 4y agoNo, it's close to the core internals of an OLAP database. The "operators" it can execute in question are things like filter, join, aggregate, group by, projection (select), things of that nature. It makes sure to use available resources like SIMD and multithreading to do that efficiently. If you built a SQL parser -- and also the glue to create query plans from that -- you could attach it to Velox to do all that on some data source, for example. But you'd still need a storage layer (disk, s3) and also some kind of higher layer if you wanted to use multiple computers for a complete database. The query execution engine is a critical component, however.