4 ms·
Did you check if this leads to any actual benefits? If so, how did you benchmark it?
by altmanaltman 7mo ago
Did you check if this leads to any actual benefits? If so, how did you benchmark it?
- pranabsarkar 7mo agoUpdate — ran a real bench on the live cluster (59 memories: 8 canonical facts × 3-4 paraphrases + 6 seeded contradictions + 20 distractors). Numbers: duplicates per query (top-10): 0.9 → 0.0 top-result correct: 75% → 87.5% 11 consolidations in 80ms conflicts detected: 0 of 6 seeded ← this one matters Turns out conflict detection runs on graph edges, and /v1/remember doesn't auto-extract entities — so contradictions sit there invisibly until you explicitly call relate. That's a UX gap, not a missing feature, but it breaks the "drop memories in, get contradictions out" mental model. Filed as issues #1 and #2. Dataset + script + raw results: https://gist.github.com/spranab/49c618d3625dc131308227103af5aadd https://gist.github.com/spranab/49c618d3625dc131308227103af5.... Honest benches surface the kind of thing demos hide; thanks for pushing.