Google Research's RRSI Guide: Building Self-Improving AI Agents That Don't Cheat Themselves

By Sana Hassan | October 8, 2026

Introduction

In this tutorial, we implement RRSI (Regularized Recursive Self-Improvement), a method that allows an LLM agent to rewrite its own harness, prompts, tools, memory, control flow, and sub-agents around a frozen model—without letting the harness overfit to the tasks it evolves on. This distinction matters more than ever in 2026, as agentic systems increasingly operate in open-ended environments where naive self-optimization can silently degrade generalization.

The full RRSI loop drafts edits with Claude Opus on Vertex AI and scores them against Docker-based benchmarks, which is not something a free notebook can run end to end. However, the part of RRSI that actually carries the paper's core idea—the rules that decide which proposed edits to keep—is plain Python. That is what we drive directly here.

We install the package from the official repository, walk through its estimator, its calibrated noise band, both branches of its selection algorithm, its annealed edit budget, its deterministic leakage screen, and its edit history. We then plug a simulated agent into RRSI's own Domain interface. Because we built the simulated environment ourselves, we know the true effect of every edit, which lets us audit RRSI's decisions against ground truth and compare them with an unregularized search that simply keeps whatever scores highest.

Setting Up the Environment

We begin by importing the standard library modules RRSI's decision logic depends on: filesystem access, JSON handling, math for calibration, deterministic randomness, subprocess control, and statistics for the estimator.

import os
import sys
import json
import math
import copy
import random
import tempfile
import textwrap
import traceback
import subprocess
import statistics as st
from pathlib import Path

RESULTS = {}


def banner(title):
    print("\n" + "=" * 78)
    print(title)
    print("=" * 78)


def section(name):
    def wrap(fn):
        def run(*a, **kw):
            banner(name)
            try:
                out = fn(*a, **kw)
                RESULTS[name] = out if isinstance(out, str) else "ok"
                return out
            except Exception as e:
                RESULTS[name] = f"SKIPPED / FAILED -> {type(e).__name__}: {e}"
                print(f"\n[!] {name} did not complete")
                traceback.print_exc()
        return run
    return wrap

Why Regularization Is the Point

The central insight behind RRSI is deceptively simple: an agent that can rewrite itself will, if left unconstrained, optimize for the exact tasks it is evaluated on. This is the agentic equivalent of a student who memorizes practice exams instead of learning the subject. The paper's regularization mechanisms—noise bands, leakage screens, and annealed budgets—exist to prevent that failure mode.

By 2026, this concern has moved from theoretical to practical. Teams shipping production agents routinely encounter harnesses that score brilliantly on their internal eval suite while collapsing on user traffic. RRSI's selection rules offer a principled way out.

Next Steps

The remainder of this tutorial walks through each component of RRSI's selection logic in sequence:

1. The estimator — how RRSI estimates an edit's expected effect from noisy benchmark scores.
2. The calibrated noise band — how the system decides whether a score improvement is real or just variance.
3. Selection branches — the two paths in RRSI's algorithm for accepting or rejecting edits.
4. The annealed edit budget — how the system gradually tightens the criteria for accepting self-modifications over time.
5. The leakage screen — a deterministic check that catches edits attempting to game the evaluation.
6. Edit history — how RRSI maintains an audit trail of every self-modification for later inspection and rollback.



Each section includes runnable code and annotated output, so you can trace exactly why RRSI accepts or rejects any given proposal. By the end, you will have a working mental model of how to build self-improving agents that improve honestly—and how to catch the ones that don't.

via MarkTechPost