Don't Just "Throw Adam at It": Misunderstanding Adam Will Cost You

I spent weeks wrestling with a particularly hard reinforcement learning problem. The research papers showed similar agents mastering comparable tasks, yet our model was dead in the water.

I simplified the architecture. I added layers. I removed layers. I swapped LSTMs for Transformers, added attention mechanisms, and then removed them. I rebuilt the input features at least 20 times. I even experimented with exotic memory architectures. In pathological hyperparameter search mode, I burned thousands of GPU hours—and still couldn’t reproduce the results.

Nothing.

The fix, when I finally found it, was humiliating: I changed β₂ from 0.999 to 0.95 and reduced β₁ from 0.9 to 0.5.

# The "humiliating" fix that actually worked:
optimizer = optim.AdamW(
    model.parameters(), 
    lr=3e-4, 
    betas=(0.5, 0.95), # The magic numbers
    eps=1e-4           # Preventing division by zero in RL
)

Most deep learning engineers, myself included, default to a simple mantra:

Just use AdamW with the default parameters and set the learning rate to 3e-4.

I don’t say this to criticize—I’ve fallen into the same trap. Yet if you rely on this default, you’ll almost certainly treat the optimizer and learning rate as the only dials worth turning.

As a test, I prompted GPT-5.6 with "create a training script for a deep learning model".

Unsurprisingly, it happily generated:

CONFIG = {
    "lr": 3e-4,
    "weight_decay": 1e-5,
     ...
}
....
optimizer = optim.AdamW(
   model.parameters(),
   lr=CONFIG["lr"],
   weight_decay=CONFIG["weight_decay"]
)

In Karpathy we trust.

3e-4 is the best learning rate for Adam, hands down.

— Andrej Karpathy (@karpathy) November 24, 2016

Nope.

The problem with this dangerous assumption is that engineers gloss over a critical aspect of training, treating the optimizer as a solved problem.

You might get away with it—the defaults are sensible for many cases. But in the long run, especially when you encounter a non-stationary or difficult optimization landscape, you will fail. Hard.

By 2026, as models grow in complexity and are deployed in ever more dynamic environments, understanding Adam’s internals isn’t optional—it’s essential for extracting reliable performance.

In this article, I’ll cover:

1. How Adam actually works
2. Where Adam fails spectacularly
3. The intuition behind modifying the defaults

This isn’t another generic review of parameter optimization.

This is why your incorrect assumption of “just use Adam” is wrong—with the mathematical intuition as to why.

Who this is for: You use Adam as your optimizer of choice with a surface-level (or no) understanding of how it works and where it fails.

How Adam actually works

Vanilla stochastic gradient descent (SGD) applies the same learning rate to every parameter:

$\theta_t = \theta_{t-1} – \alpha \cdot g_t$

This is the equation most practitioners know. If you have a CS degree, you may have implemented it from scratch.

Multiply by the gradient, subtract from the parameters, repeat.

Adam improves on SGD by maintaining per-parameter learning rates—it adapts based on the first moment (mean) and second moment (uncentered variance) of past gradients. The key hyperparameters β₁ and β₂ control the decay rates for these moments. Defaults of β₁=0.9 and β₂=0.999 work reasonably well for many vision and language tasks, but they assume stationary gradient statistics—an assumption that breaks in reinforcement learning, adversarial training, and other non-stationary settings.

In my RL case, the high β₂ (0.999) meant that the moving average of squared gradients held onto old information too long, failing to adapt quickly to a changing reward landscape. Dropping it to 0.95 made the optimizer respond within ~20 steps instead of ~1000 steps—the fix that saved my model.

Understanding these failure modes is the first step to moving from "just using Adam" to using it effectively. In later sections, we’ll explore the full math behind Adam’s update rules, diagnose specific failure points, and develop intuition for when to deviate from the defaults.

via Towards Data Science