When Should LLM Agents Forget? Memory Filtering Under Query-Critical Resolver Errors
Author: Samarth Vinayaka
arXiv: 2611.17338 [cs.AI]
Submitted: 30 August 2026
Comments: 12 pages, 2 figures
DOI: https://doi.org/10.48550/arXiv.2611.17338
Abstract
LLM memory systems often delete facts classified as stale. We show that this hard-filtering step can reduce downstream accuracy when resolver errors delete query-critical facts. Across three models, a controlled sensitivity intervention moves hard-drop from gains of +0.186 to +0.230 at perfect resolution to losses of -0.176 to -0.181 at the lowest tested reliability. Paired targeted interventions show that false-drop causes significantly greater degradation than false-keep, while a non-deleting baseline can still fail when incorrect labels steer generation. On a frozen 102/102 calibration/test split, a simple deployment-matched gate reaches 0.655 accuracy versus 0.641 for always hard-drop and 0.662 for test-informed oracle selection. Global-accuracy crossovers require gold labels and are model- and task-specific diagnostics, not deployment thresholds. Deletion should instead be treated as cost-sensitive: validate query-critical false-drop risk on matched data, or default to label-free retention when such validation is unavailable.
Summary
As agentic memory systems mature in 2026, this paper challenges the default practice of aggressively purging "stale" facts from LLM memory. Its central finding is that hard filtering is only beneficial when the resolver that classifies memory is highly reliable. When resolver errors cause query-critical facts to be dropped (false-drops), downstream accuracy can fall sharplyβand these losses outweigh the harm from retaining incorrect facts (false-keeps).
The author recommends treating deletion as a cost-sensitive decision: validate query-critical false-drop risk on matched deployment data, or, when such validation is unavailable, default to label-free retention.
Key Takeaways
- Hard-drop is reliability-dependent. Gains at perfect resolution (+0.186 to +0.230) reverse into losses (-0.176 to -0.181) at the lowest tested resolver reliability across three models.
- False-drop hurts more than false-keep. Paired targeted interventions show false-drop causes significantly greater degradation.
- Not deleting is not automatically safe. A non-deleting baseline can still fail when incorrect labels steer generation.
- Deployment-matched gating helps. On a frozen 102/102 calibration/test split, a simple deployment-matched gate reaches 0.655 accuracy, versus 0.641 for always hard-drop and 0.662 for a test-informed oracle.
- Global-accuracy crossovers are diagnostics, not thresholds. They require gold labels and are model- and task-specific, so they cannot serve as deployment thresholds.
Practical Guidance
- Validate before deleting. Measure query-critical false-drop risk on data matched to the deployment setting.
- Default to retention when validation is unavailable. Label-free retention is the safer fallback.
- Treat memory filtering as cost-sensitive. Weigh the asymmetric costs of false-drop versus false-keep rather than applying blanket staleness rules.
- Use deployment-matched gates. Calibrate filtering decisions on representative data rather than relying on global-accuracy crossover points.
Subjects
Artificial Intelligence (cs.AI)
Cite As
arXiv:2611.17338 [cs.AI] (or arXiv:2611.17338v1 [cs.AI] for this version)
via ArXiv AI
