Single-hop recall
Direct facts from recent or distant sessions. Tests faithful storage and retrieval, the baseline for any memory system.
Cat1 · Dakera v0.11.107
86.9%282 questions
Dakera on LoCoMo:88.2% Recall@20
The full methodology and per-category breakdown. 10 conversations, 1,536 evaluated questions, no LLM in the retrieval path. A question counts as a hit when at least one of its gold evidence dialogue turns is among the top 20 retrieved memories, scored deterministically with no LLM judge.
- 1,536of 1,540 questions evaluated; the adversarial category is excluded
- 10long-term conversations, up to ~35 sessions each
- Top 20retrieved memories checked against the gold evidence
- v0.11.107the Dakera version measured, September 2026
Four question categories
LoCoMo tests four distinct recall challenges. Three score above 85%; temporal inference is our hardest category and an active area of improvement.
88.2%Overall Recall@20 across 1,536 questions from 10 conversations. No LLM in the retrieval path; adversarial category excluded.
Multi-hop reasoning
Questions that need inference across two or more stored memories. Tests whether recall chains compound correctly.
Cat2 · Entity graph traversal enables multi-hop inference
85.4%321 questions
Temporal inference
Sequences, durations and before-and-after relationships across stored memories.
Cat3 · Hardest category industry-wide, and an active roadmap item
73.9%92 questions
Open-domain recall
Free-form questions spanning multiple topics, entities and sessions. Tests the breadth of the retrieval system.
Cat4 · Improved by hybrid retrieval and knowledge graph traversal
91.0%841 questions
What 88.2% means for your agent
Benchmark scores translate directly to agent behavior. Recall failures make agents repeat questions, forget context and give inconsistent answers.
<10ms
Production latency
P99 query latency under 10 ms, with no LLM in the retrieval path.
- Version
- v0.11.107
- Index
- Warm, in-memory, 20k vectors
- Query
- top_k=20, single-threaded
- Host
- Idle 8-core x64
- Server-side P99
- Under 10 ms
- End to end, incl. client
- ≈16 ms
Latency scales with index size.
88.2%
Fewer repeated questions
At 88.2% recall, your agent remembers what the user told it, across sessions. No “as I mentioned earlier” failures.
91.0%
Long-horizon context
91.0% open-domain recall means agents carry context across days, weeks and months, not just within a single conversation.
What 88.2% actually means
The benchmark dispute
Other vendors claim higher LoCoMo scores: Zep at 94.7%, Mem0 at 92.5%. Both numbers are actively disputed: Mem0 contests Zep’s methodology, and Zep disputes Mem0’s. The disagreement is about what the benchmark actually measures.
Dakera’s 88.2% is an evidence-match Recall@20 figure: a question counts as a hit when at least one of its gold evidence dialogue turns is among the top 20 retrieved memories, and scoring is deterministic, with no LLM judge. There is no LLM in the retrieval path either: retrieval is hybrid HNSW and BM25 fusion followed by a non-LLM temporal re-ranker.
That measures a different thing than the QA-accuracy scores some vendors report, where an answering LLM generates a response that is then judged, so the numbers are not directly rankable against each other. The full methodology is in the benchmark methodology post.
94.7%Zep’s claim, contested by Mem0
92.5%Mem0’s claim, contested by Zep
How Dakera compares
Other vendors publish LoCoMo results using different protocols and, in most cases, a different metric. Dakera’s 88.2% is Recall@20 (does retrieval surface the answer?); the figures below are QA-accuracy (does an answering LLM produce the correct answer?). They measure different things and are not directly rankable: the table is context, not a leaderboard.
| System | LoCoMo score | Metric and methodology |
|---|---|---|
| Dakera v0.11.107 | 88.2% | Recall@20 (deterministic evidence-match retrieval recall, no LLM judge) · no LLM in the retrieval path |
| Letta Filesystem | 74.0% | QA-accuracy · Letta’s published evaluation (letta.com/research) |
| Zep (corrected, 2025) | 75.1% | QA-accuracy · Zep’s own corrected score after fixing evaluation errors |
| Mem0 (Letta independent eval, 2025) | 68.5% | QA-accuracy · third-party evaluation by Letta |
Mem0’s self-reported 2026 score (92.5%) and Zep’s 2026 claim (94.7%) use different models and evaluation setups, so they are not directly comparable. See the methodology for details.
How we evaluate
We publish the evaluation in full, so you can audit exactly how the number is produced and run it again yourself.
1,536 questions, not a sample
All scores use the public LoCoMo release: 10 simulated long-term conversations of up to ~35 sessions each. 1,536 of the 1,540 QA questions across four categories are evaluated (the adversarial category is excluded), and we do not use sampled subsets. Two percentage points on a 100-question eval is statistical noise; on 1,536 questions it is a real signal.
Version: Dakera v0.11.107, evaluated September 2026.
Ingest conversations
Each LoCoMo conversation is ingested through
POST /v1/memory. Session boundaries are preserved, with no preprocessing or summarization.Run recall queries
Each of the 1,536 evaluated questions is issued as a
POST /v1/recallquery. Dakera returns its top 20 memories using HNSW and BM25 hybrid retrieval with a non-LLM temporal re-ranker.Score by evidence match
Each question lists the gold evidence dialogue turns that support its answer. It is a hit when at least one of them is among the top 20 retrieved memories (Recall@20). The check is deterministic, with no LLM judge; we use the standard LoCoMo evaluation prompt.
Aggregate and report
Scores are aggregated per category and overall. 88.2% is the Recall@20 mean across the 1,536 evaluated questions; category-level scores are approximate (±1%) because of question distribution variance.
The full evaluation protocol is documented in our benchmark methodology post. Questions about the numbers? Open an issue on GitHub.
The memory layer that documents its numbers
Dakera is self-hosted and open core. Run it in 5 minutes, measure recall on your own data, or reserve a spot on the managed cloud.
Methodology: the full write-upDakera vs Mem0Dakera vs ZepDakera vs LettaAll comparisons