3 ms·
TinyLFU is kind of magical--most of the win you'd get from expensive tracking of key frequencies, in much less space. The bloom filter in front seems like a cle
by twotwotwo 7y ago
TinyLFU is kind of magical--most of the win you'd get from expensive tracking of key frequencies, in much less space. The bloom filter in front seems like a clever hack.
Not an expectation/criticism/etc. of Ristretto, but out of curiosity, does anybody know of work about making the cache aware of the expected cost of a cache miss for each key?
To spell out what I mean: if I'm sending a query result to the cache, I could also pass along how long the query took to run. Then, in theory, the cache could try to drop keys with low (expected number of fetches * expected cost per miss), not just low (expected number of fetches). Of course that's rough/imperfect, and for some caching scenarios the miss cost doesn't vary by key.
I guess mostly the question is if it's enough of a help to justify the metadata+complexity, which I guess requires a trace of gets/sets from something that has miss costs that vary.
The intuition is just that now that we're pretty good at optimizing hit rate, maybe expanding the problem to "minimize total miss cost" could be an interesting direction for the use cases where there's a difference.
- mrjn 7y agoThat would indeed be an interesting research area.
- NovaX 7y agoThere is not a lot of research in this area and I briefly outlined my understanding of the current state when responding to a github issue: https://github.com/ben-manes/caffeine/issues/345#issuecomment-530571533 https://github.com/ben-manes/caffeine/issues/345#issuecommen...
- twotwotwo 7y agoOh, fascinating. My intuition re: size was that you'd look at worth per byte for something like query results, a whole different kind of thing from the CDN use case that's been studied. Makes me wish I could justify adding instrumentation to our 'memoize' wrappers at work, although our caching doesn't really match the use case for something like Ristretto or Caffeine. (Little enough stuff and the latency of a networked cache is fine for us, so "just set aside lots of RAM on some cache servers" works well enough.)