5 ms·

> the Query Graph Model (QGM) representation is quite abstract and hardcodes many properties, making it exceptionally difficult to understand. Its claimed exten

by refset 3y ago

> the Query Graph Model (QGM) representation is quite abstract and hardcodes many properties, making it exceptionally difficult to understand. Its claimed extensibility is also questionable. I don't know much about the context, but it was interesting to note that Materialize scrapped their QGM code last year: https://github.com/MaterializeInc/materialize/pull/17139 https://github.com/MaterializeInc/materialize/pull/17139 Also, a couple of interesting projects in the IR space: - https://substrait.io/ https://substrait.io/ is a cross-language serialization for Relational Algebra - https://www.lingo-db.com/ https://www.lingo-db.com/ is an MLIR-based (LLVM) query engine described extensively in this paper https://db.in.tum.de/~jungmair/papers/p2485-jungmair.pdf?lang=en https://db.in.tum.de/~jungmair/papers/p2485-jungmair.pdf?lan...

cmrdporcupine 3y ago
> - https://substrait.io/ https://substrait.io/ is a cross-language serialization for Relational Algebra This is awesome.
- simicd 3y ago
  Agree, substrait is a really cool project! Related: if you like substrait you might want to check out datafusion too. The project is a query execution engine built on top of Apache Arrow (incl. SQL parser, query planner & optimizer, execution engine, extensible user defined functions, among others) and it implements a substrait provider and consumer: https://github.com/apache/arrow-datafusion/tree/main/datafusion/substrait https://github.com/apache/arrow-datafusion/tree/main/datafus...
  - cmrdporcupine 3y ago
    In general I'm not interested in analytics/OLAP, but in more OLTP type workloads, so the Arrow stuff and that whole world is of not much interest to me.
leiysky 3y ago
A fun fact, I was there when I first see them going to put it in use. That’s a pity they finally gave it up. https://github.com/MaterializeInc/materialize/issues/8073 https://github.com/MaterializeInc/materialize/issues/8073

by refset 3y ago

cmrdporcupine 3y ago

> - https://substrait.io/ https://substrait.io/ is a cross-language serialization for Relational Algebra This is awesome.

simicd 3y ago
Agree, substrait is a really cool project! Related: if you like substrait you might want to check out datafusion too. The project is a query execution engine built on top of Apache Arrow (incl. SQL parser, query planner & optimizer, execution engine, extensible user defined functions, among others) and it implements a substrait provider and consumer: https://github.com/apache/arrow-datafusion/tree/main/datafusion/substrait https://github.com/apache/arrow-datafusion/tree/main/datafus...
- cmrdporcupine 3y ago
  In general I'm not interested in analytics/OLAP, but in more OLTP type workloads, so the Arrow stuff and that whole world is of not much interest to me.

leiysky 3y ago

A fun fact, I was there when I first see them going to put it in use. That’s a pity they finally gave it up. https://github.com/MaterializeInc/materialize/issues/8073 https://github.com/MaterializeInc/materialize/issues/8073