2 ms·
If you are willing to give up range scans and constrain yourself to fitting all keys in memory there is a lot you can do. See SILT (www.cs.cmu.edu/~dga/papers/s
by arielweisberg 14y ago
If you are willing to give up range scans and constrain yourself to fitting all keys in memory there is a lot you can do. See SILT (www.cs.cmu.edu/~dga/papers/silt-sosp2011.pdf) and Bitcask (downloads.basho.com/papers/bitcask-intro.pdf).
Since this is a cache I really dig skipping any kind of cleanup/compaction step for deleted/expired keys.
I played around with a similar thing except as a K/V store and the performance and density is pretty amazing. With a 64 byte key and 1.5k value (compressed from 2k) I was getting 85k inserts/sec and several hundred thousands reads/sec with a quad-core Sandy Bridge i5 and a 128 gigabyte Crucial M4 on SATAII.
- silasb 14y agoWonder how much of the SATAII was the bottleneck, my guess is quite a lot. You should give your test another go with SATAIII.
- justincormack 14y agoFitting all keys in memory is a pretty harsh constraint though. With your 64byte to 1.5k you are talking about a factor of 20, so for a 1TB SSD you need 64GB RAM, which costs about as much as the SSD.
- efuquen 14y ago"so for a 1TB SSD you need 64GB RAM, which costs about as much as the SSD." In my experience this is not true. A 1 TB SSD, even commodity hardware, is pretty expensive. And that's not even talking about high performance SSD's like Fusion IO, where you're now talking about 10K+. Where do you see 64 GB of RAM costing that much?
- justincormack 14y agoMicron unveils 1TB SSD for $600, which is about what 4x16GB ECC RAM is... https://www.computerworld.com/s/article/9235277/Micron_unveils_its_first_1TB_SSD_for_under_600 https://www.computerworld.com/s/article/9235277/Micron_unvei... Fusion IO is probably as expensive as RAM, so only makes sense for apps that really need persistence, ie databases.