4 ms·

Curious how semantic cache works,are cache hit based on embeddings, exact prompt normalization, how do you avoid serving stale incorrect cache..

• gurveer51 2mo ago
  for semantic cache, there is configurable similarity threshold. using it you can control the incorrect cache serving.
  • iwasinnam 2mo ago
    [flagged]