3 ms·
Hi, author here :) In general, many databases have attempted something similar- but because "batch query" engines are so monumentally different than incremental
by maor10 3y ago
Hi, author here :)
In general, many databases have attempted something similar- but because "batch query" engines are so monumentally different than incremental engines, it becomes a very tough task and ends up being a hill of patches- which leads to many limitations.
The easiest example I can think of is the direction of the flow of data. Postgres, for example, works from "top to bottom", where you start with the result node which runs the node underneath it and so on. Each node continues requesting information from the node underneath it until it returns Null, meaning it finished. This works very well with a snapshot of the data- but if you want to maintain a high throughput with a constant stream of changes, you're going to want to work in the opposite direction- this would mean either overhauling the very way Postgres works, or adding a completely different engine into Postgres.