That is not true. RAM is cheap (terabytes per node is not uncommon). Moreover, a common pattern for kdb+ databases is to have two parts: a real-time database (r

That is not true. RAM is cheap (terabytes per node is not uncommon). Moreover, a common pattern for kdb+ databases is to have two parts: a real-time database (rdb) that is typically memory backed and a historical database (hdb) that is serialized to disk (mmap'd for near-RAM speed). In practice, most data fetching is sequential (cache friendly), so you get good performance regardless. kdb+ is commonly used in the financial industry, where data volumes are large. The language itself is extremely minimal, so the learning curve is actually much lower than most languages (it forces you to write in a "vectorized" style, which is not too different than how numpy/R/matlab works). I was able to get up to speed in a couple of days. Source: Used to work with kdb+ (still do sometimes). Not a shill (I don't even like k/j/q/APL).