3 ms·

Any data on better outputs and/or token saving?

• divitsheth 3mo ago
  We’re working on evals now. We ran a small preliminary LoCoMo test with a 2k-token retrieval budget: Almanac: 55.7% BM25: 51.8% Supermemory: 47.6% Mem0: 60.6% LoCoMo is not quite our use case. It tests conversational memory, while Almanac is more for coding agents trying to find their way around a codebase. These results show output quality at the same token budget, not token savings yet. We’re working on coding agent evals that should be more representative.
  • pwillia7 3mo ago
    Cool thanks I will keep an eye on it!