Princeton Researcher Proposes Recurrent Looped Transformer (RLT) That Carries Decoder State Across Every

In most decoder-only LLMs, nothing computed at the last layer of token t feeds the first layer of token t+1; positions communicate only through attention over cached keys and values. A Princeton researcher's technical report, Recurrent Looped Transformer (RLT), proposes closing that loop.

The decoder's final hidden state and its layerwise sliding-window attention (SWA) cache are carried into the next token, across both prompt and response, with no reset at the boundary. The proposed research is a design specification: it defines the architecture, execution schedules, and RL replay contract, and explicitly reports no measured efficiency, reasoning quality, or scaling results.

As of 2026, this work arrives amid growing interest in test-time compute and recurrent depth as alternatives to simply scaling parameter counts. RLT sits alongside other looped and recurrent-depth proposals that aim to extend effective compute without proportionally increasing weights.

How RLT Is Built

RLT pairs a causal encoder with a recurrent decoder. The encoder processes tokens in parallel under a causal mask and produces representations e_t, from which key-value memory M_≤t is projected; memory groups can be shared across decoder layers (G = 1) or kept layer-specific (G = L_D).

The decoder holds the recurrence. Its complete state is H_t = (s_t, C_t^D), where s_t is the final decoder output and C_t^D holds the retained SWA keys and values at every decoder layer. For each token, a gated merge combines e_t with the previous output s_t−1, then each decoder block runs causal SWA over decoder activations, cross-attention to encoder memory, and an FFN. The window W includes the current token, so at most W − 1 historical entries per layer are retained. The next-token distribution is read from s_t, and initialization happens once before BOS with a learned start state s* and an empty cache.

The reference tied configuration uses 48 encoder and 48 decoder layers with compatible attention and FFN weights shared between them. Each token therefore executes 96 logical blocks, though decoder blocks add cross-attention, so per-block FLOPs are not equal. Zhang calls this parameter reuse, not activation copying.

The Three Design Principles

According to the report, RLT is organized around three design principles:

1. Explicit temporal state carry-over. The recurrent loop transmits the decoder's full state—final output plus layerwise SWA caches—from token to token, rather than discarding it at each position boundary.
2. Bounded context per layer, unbounded temporal depth. Sliding-window attention caps the number of retained entries per layer at W − 1, while the recurrent path can in principle extend across arbitrarily many tokens.
3. Parameter reuse over parameter growth. The tied configuration shares attention and FFN weights between encoder and decoder layers, increasing effective depth through looping rather than by adding unique parameters.

The report frames RLT as a specification for further study. It does not claim benchmark results, throughput improvements, or scaling curves. Whether the recurrent state carry-over yields better reasoning or efficiency remains an open empirical question.

via MarkTechPost