Benched cold start recently: 19k LongMemEval sessions laid down in the real ~/.claude and ~/.codex layouts, 100 questions whose answer sits in exactly one sessi

Benched cold start recently: 19k LongMemEval sessions laid down in the real ~/.claude and ~/.codex layouts, 100 questions whose answer sits in exactly one session, scored by whether that session comes back (bias: I built deja, one of the six) deja: 29s to index, 24ms query, 18/100 hit@1, 67 found@50. Plain BM25, no vectors agentmemory: 95s import, 14 hit@1, 65 found@50, plus a worker and engine on four ports MemPalace: ~3h mining, 2.6s query, 14 hit@1 CASS: 56m index, every NL query fails with "query fuel exhausted" on the release build (fixed on their main) claude-mem: no-op out of the box, only records forward from install funes: the documented 1 min first pass indexed 189 of 19k sessions (0/100); full index still embedding, ~3 sessions/s Numbers look low because 19k sessions is brutal; on the standard 500-session LongMemEval-S the same BM25 gets ~85% hit@1 The funny thing is BM25 basically ties embeddings here at 1/100th the cost. The real cliff is reranking (found@50 67 vs hit@5 35) and staleness. Vector search has zero concept of "superseded info" only fix I found was letting explicit user corrections outrank the transcript. Repro scripts and corpus: https://vshulcz.github.io/deja-vu/guide/day-zero.html https://vshulcz.github.io/deja-vu/guide/day-zero.html @skeledrew: cross-agent across 23 harnesses, but yeah, it's an index over logs, not a source of truth :)