3 ms·
Certainly. Every cache (including a semantic cache like GPTCache) needs an eviction policy to prevent the size of the cache from exploding. A single line is the
by fzliu 4y ago
Certainly. Every cache (including a semantic cache like GPTCache) needs an eviction policy to prevent the size of the cache from exploding. A single line is the smallest unit a cache can hold - for a semantic cache, that means a single pair of unstructured data such as text/text or text/image.