LongMemEval-S: 96.0% with the official judge, and what top-5 vs top-20 retrieval changes [R]
We ran all 500 LongMemEval-S questions through our memory retrieval (Papr) using the official answer prompt and the unmodified gpt-4o-2024-08-06 judge. Per-question answers and judge labels are public, so the official metrics script reproduces the numbers. Results For comparison, Chronos reports 95.6% using full context and multi-turn agents, Hindsight reports 94.6% at 43.6k tokens per question (with a different judge), and gbrain reports 90.6%. Things I found useful Retrieval recall is close to saturated. All evidence sessions are in the top 5 for 460 of 470 answerable questions (97.9%). Most remaining errors are the reader, not retrieval. Top 20 vs top 5 is a trade, not a free win: 11 gained, 8 lost, net +3. Temporal reasoning went 123 → 130 of 133. Abstention stayed flat (27/30 both). A small reader gets most of the way: gpt-5-mini reaches 94.4% on the same retrieval. Caveats Used Papr-Embed 0.6B embedding model (open source and open weight), gemini flash 2.5 reranker (also tested J
Related Stories
Anthropic CEO Amodei admits "there's a huge uncertainty in what's going to happen next year" days before a record IPO
The Decoder · November 14, 2026
Gemini 4 Carbon takes the MathArena crown but costs 15 times more than GPT-6.1 Sol
The Decoder · November 14, 2026
Anthropic says outdated prompting habits are slowing down Claude Opus 5.5
The Decoder · November 14, 2026