Meta FAIR's Self-Improving Memory Scores 56.0 on Continual Learning Benchmark
Meta FAIR has introduced Self-Improving Memory, a method for AI agents that manage their own memory. The model first discovers its own memory instructions, then trains those skills into its weights. Co-authors include researchers from the University of Washington and Johns Hopkins University.
TL;DR
- Method: The base model is Kimi-K2.6, a 1T-parameter MoE with 32B active parameters and a 256K context window. Distillation adds a LoRA adapter (rank 64) on attention projections.
- Runs on: Training hardware is not disclosed. Serving Kimi-K2.6 requires a multi-GPU node; Vast.ai recommends 8x H200. Native INT4 weights are available.
- Performance: Self-discovered memory prompts average 56.0 across 3 tasks, versus 44.6 for the best human-written prompts.
- Best result: Codebase, at 64.3 with optimized prompts—a gain of 20.6 over the best hand-written harness.
- Worst result: Spectrum distillation adds only 2.9 points (46.5 vs. 43.6 for SFT on generic runs).
What Is Self-Improving Memory?
Self-Improving Memory is a two-stage method from Meta FAIR that teaches an agent model to manage file-based memory. The agent works through a stream of subtasks. Each subtask starts with an empty context; only a persistent memory directory, edited with file tools, carries over.
Two prompts control that memory. OFFLOAD updates memory after a subtask. LOAD retrieves notes before the next one. Generic versions leave the strategy to the model. A pilot study in the research post found that no single hand-written strategy wins every...
via MarkTechPost
