3 ms·
I'm working in something like this but only in memory (because I'm building a relational language). I have wonder how much better could be row or columns store
by mamcx 10y ago
I'm working in something like this but only in memory (because I'm building a relational language).
I have wonder how much better could be row or columns store if only for the in-memory data build typically in a program execution. I wonder if exist some data about this - because I'm too early stage to have meaningful data for benchmarks and my intuition is that column-store could be better long-term because searching for few columns dominate all other things (and joins could be better?), but in other aspects, like iterating, things not look good.
BTW, this is in F#
- Everlag 10y agoThe 'optimal' store really depends on the type of data and queries you're targeting. [0] is probably a good place to start for comparisons. I'd also recommend the other reading in my previous post, but that is much longer. Sequential scans form the core of most column store queries, why would iteration be an issue? Do you mean iterations considering multiple fields of a structure or similar? In that case, you should be able to break that into a series of efficient, per-column queries which can be lazily materialized. [0] http://db.csail.mit.edu/projects/cstore/abadi-sigmod08.pdf http://db.csail.mit.edu/projects/cstore/abadi-sigmod08.pdf
- mamcx 10y agoYeah, normal iterations that select most/all columns like for row in customers and similar... However I see that kdb+ is column-store (http://www.timestored.com/kdb-guides/kdb-database-intro http://www.timestored.com/kdb-guides/kdb-database-intro) and supposedly the performance is ok?