4 ms·
A high performance caching library for Java 8
- NovaX 11y agoI've been doing a lot of work on a new optimal eviction policy. It out performs or matches the current gold standard, without requiring the cache holds evicted keys as those policies do (doubles or triples the physical size). Instead it uses a 4-bit CountMinSketch to record the historic frequency and two LRU queues to balance temporal workloads (high/low frequency + high time locality). This approach is simpler, fast, and much more memory efficient. By moving to a policy that knows which entries are hot, a fast path read could be added to skip policy maintenance for the hottest entries. This could improve performance by 2-3x on a Zipf workload (typical) and has minimal impact on the hit rate. We're hoping to publish a paper on this new policy. I'm currently integrating it into the cache (but already implemented in the simulator).