2 ms·
"GPTCache employs embedding algorithms to convert queries into embeddings and uses a vector store for similarity search on these embeddings. This process allows
by eternalban 4y ago
"GPTCache employs embedding algorithms to convert queries into embeddings and uses a vector store for similarity search on these embeddings. This process allows GPTCache to identify and retrieve similar or related queries from the cache storage"
previously: https://news.ycombinator.com/item?id=35547426 https://news.ycombinator.com/item?id=35547426
@fzliu: can you expand on the eviction policy - "lines"?
- fzliu 4y agoCertainly. Every cache (including a semantic cache like GPTCache) needs an eviction policy to prevent the size of the cache from exploding. A single line is the smallest unit a cache can hold - for a semantic cache, that means a single pair of unstructured data such as text/text or text/image.