3 ms·
> TTL doesn’t say anything about eviction among objects that are still live, no? TTL IS eviction policy. You just store a timestamp along the value. When you a
by hknmtt 3y ago
> TTL doesn’t say anything about eviction among objects that are still live, no?
TTL IS eviction policy. You just store a timestamp along the value. When you access it, you check the timestamp and if it is stale, you delete it and return not found error(or whatever not found should return). And you have a simple bg worker that scans the cache periodically and deletes these stale entries to free up the memory.
This is the basis for all caches. You can add cost on top of this as yet another eviction policy but that only adds another worker to scan the cache for items to purge and you have to store the cost along with the entry so you are increasing the each entry by at least 4 bytes, if we're talking uint32. Not to mention that for this eviction policy, you need sorted cache because you need to know where the cut off for cost is.
Walking the entire cache to purge expired TTL entries is one thing, but keeping ordered cache is a whole another thing.
- klabb3 3y agoWhat I mean is that when cache is full you need an eviction policy, like LRU for instance. If everything fits in memory there’s not much to discuss, yes then TTL with periodic scan is reasonable, yes.
- 1a1a11a 3y agoSorry. TTL is not an eviction policy, as the others pointed out, you need an eviction algorithm to decide which objects to keep in the cache when it is full. Your mindset is about a key-value store, not kv cache, which is always full. I agree that TTL is useful, but it is NOT a replacement for the eviction algorithm. Moreover, when you have a huge multi-tenanted cache, different engineers set TTLs in different ways, some use TTLs to capture data lifetime, some use TTLs to guard around staleness, some use TTLs for legal compliance. Therefore, TTLs do not indicate eviction priority.