17 ms·
There are likely many ways of doing these things, but here is my vague recollection from the time when I worked on a C++ in-memory database. We used tree tran
by TrainedMonkey 4y ago
There are likely many ways of doing these things, but here is my vague recollection from the time when I worked on a C++ in-memory database. We used tree transformations for query planner/optimizer. The evaluator was implemented as a virtual interface that had a lot of complexity, but on most basic level it took some parameters and returned rows of data.
Optimizer/query planner worked by parsing the query as a tree of operations and then finding tree transformations which are equivalent. Most basically this involved swapping table join orders and trading between materializing and streaming operators(^0). Costing was based on histograms data histograms which essentially enabled estimating how many rows each node in the query tree is expected to return. Generally you want to reduce the number of rows you are materializing ASAP. This generally means joining tables with fewest rows first, because then you can efficiently scan larger tables. Default strategy for joining large tables was to scan smaller one and build a map of the values and then scan the other one checking the map for each record. An alternative strategy is to build a bloom filter instead of the map, but that is only more efficient if the cross-section of the join is small.
Virtual execute evaluate interface enabled a lot of custom execution strategies for selecting from row/column/external db based tables, sorting data, and tons of join strategies.
Note 0: The constraint with streaming operators is that you can't fully sort the data before next step is executed. This prohibits execution strategies such as efficient joins which require all data to be sorted.