2 ms·

>machine: 16 core 256GB (in all cases: date partition, sym index. all queries in RAM.) ... >all query data is cached in RAM (no disk access). 1) What happens w

>machine: 16 core 256GB (in all cases: date partition, sym index. all queries in RAM.) ... >all query data is cached in RAM (no disk access). 1) What happens when you run these benchmarks on a machine with only 16GB of RAM? 2) How does the KDB performance compare to doing the equivalent operations on a Pandas DataFrame (which, since these are simple in-memory operations, seems like the only fair comparison).

icsa 10y ago
> 1) What happens when you run these benchmarks on a machine with only 16GB of RAM? kdb+ uses ~ 1.2 MB of Resident RAM at startup. In addition, kdb+ storage model in memory and on disk have very little overhead (a few hundred bytes) per column over the raw binary data. The above, combined with memory mapping, allow for large databases to be queried with great performance. I query billion row futures databases on a Macbook Pro with 16 GB RAM with kdb+. > 2) How does the KDB performance compare to doing the equivalent operations on a Pandas DataFrame (which, since these are simple in-memory operations, seems like the only fair comparison). Could someone provide the equivalent Pandas code running against similar TAQ data for the queries in the following benchmark? http://kparc.com/q4/readme.txt http://kparc.com/q4/readme.txt