4 ms·
Great to hear that! No plans at the moment to support larger than RAM indices - that’s the trade-off we had to make for performance. Mind if I ask how large of
by jabo 5y ago
Great to hear that! No plans at the moment to support larger than RAM indices - that’s the trade-off we had to make for performance.
Mind if I ask how large of an index size you’re looking at?
- ddlutz 5y agoWhile you probably aren't using B+-tree indexes like databases, can you use a setup like them were you can create a "paged" data structure and use some buffer pool to bring pages in and out?
- karterk 5y agoCertainly that's possible but doing fast type ahead fuzzy search requires keeping large parts of the indexing data structure in-memory. There are, of course, many use cases where one does not need that speed. Eventually, like Redis we will probably have an option to use SSD storage smartly for these non-latency sensitive use cases. That's not our immediate focus though.
- FridgeSeal 5y agoWe have fairly large variation in index size, from ~100MB through to 30-50GB, and probably larger still in future. In the order of thousands of indexes as well. Application level sharding/merging for more consistent index size is on the list of things to do, but given the complexity of that it’s a “not before we really need to” kind of task haha.