Emergent Sparsity in Frozen Random CNN Feature Extractors for Deep Reinforcement Learning

Emergent Sparsity in Frozen Random CNN Feature Extractors for Deep Reinforcement Learning

Authors: Scott M. Norton

Submitted: 15 May 2026

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)

arXiv ID: 2607.26059

Abstract

We report a striking phenomenon: deep reinforcement learning agents trained with frozen, randomly initialized CNN feature extractors spontaneously develop extremely sparse fully-connected representations—without any sparsity-inducing objective. In the first fully-connected layer (FC1, from 3,136 units down to 64), agents compress task-relevant information through as few as 1–3 out of 64 neurons for deterministic Pong (5–11 for stochastic Pong). In contrast, trainable CNNs activate 55–64 neurons under matched conditions. We establish four principal findings:

1. Sparsity scales with task complexity. FC1 sparsity ranges from 1–11 neurons for Pong, 19–26 for Breakout, and approximately 42 for Space Invaders. Width-scaling experiments confirm this reflects underlying task structure rather than a fixed capacity fraction.


1. Within-game variation emerges. Three identical training seeds for Pong yield 5, 7, and 11 active neurons. The seed with only 5 neurons plateaus at a reward of +14, while the others reach expert performance (+18.4 and +18.7), suggesting that the usable dimensionality of the random projection bounds achievable performance.


1. Ablation confirms necessity. Removing the active neurons crashes performance across two distinct PPO implementations and four different games.


1. The information bottleneck locks in early. A sweep shows the active set stabilizes by 15–30 million environment steps, while reward only becomes positive 35–105 million steps later.



A complementary finding in Breakout shows frozen and trainable CNNs reach competitive rewards via structurally different bottlenecks: frozen agents use 17–25 active neurons (participation ratio ~10–14), while trainable agents use 51 (participation ratio ~3.6).




Finally, wherever input dimensionality dwarfs intrinsic task dimensionality, gradient descent on a frozen random projection may reveal the effective rank of the underlying problem without explicit sparsity machinery.




---




Comments: 24 pages, 4 figures

Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)

Cite as: arXiv:2607.26059 [cs.LG]

via ArXiv LG