Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, andNEW⭐7

Recurrent depth retrofit boosts pretrained models' iterative reasoning, beats scratchpads, extends beyond learned depth, at two parameter budgets.