3 ms·
Very cool. I wonder if the pluggable storage engine also supports query pushdown. For example, if you do SELECT ... FROM mytable WHERE key = 42, then if this i
by atombender 3y ago
Very cool. I wonder if the pluggable storage engine also supports query pushdown.
For example, if you do SELECT ... FROM mytable WHERE key = 42, then if this is a Postgres table, you want the key predicate to be pushed down to Postgres so you don't need to scan the whole table. Same for ORDER BY, joins, LIMIT, OFFSET. For joins between foreign tables from different sources, you'd want the optimizer to be aware of table/column stats in order to pick the right join strategy.
Sqlite has a "virtual table" mechanism where you can plug in your own engine. But it's relatively simplistic when it comes to pushdown.
- zacblanco 3y agoWe have some of this functionality in Presto (https://github.com/prestodb/presto https://github.com/prestodb/presto), but it takes fair bit of work to implement it for all the different backends.
- maxxen 3y agoIt does push down filters! Not sure if other nodes are pushed down, I think for now the database boundary is at the scan node, but you could add additional optimizer/rewrite rules in the postgres/sqlite/mysql extensions to push down other parts of the plans when applicable.
- CSDude 3y agoI'd also love to write storage plugins with Go or Rust, not just C++