In this tutorial, we explore cuDNN Frontend's Graph API from below the framework layer. We describe a computation as a graph of operations, let cuDNN select an engine to run it, and then take control of that choice ourselves.

Every kernel we build here follows the same pattern: declare tensors by their dimensions and strides, chain operations onto them, run the five-step build pipeline (validate, build operation graph, create execution plans, check support, and build plans), then execute against a variant pack of pointers.

We run everything on a single Colab GPU, checking each result against a PyTorch reference so we can see both that the fusion is correct and what it costs. The topics build on each other, moving from a single fused convolution to autotuning across engine configs, FP8-style epilogues, attention, plan serialization, dynamic shapes, and CUDA graph capture.

import os
import sys
import glob
import math
import time
import ctypes
import traceback
import subprocess
RESULTS = {}
def banner(title):
print("\n" + "=" * 78)
print(title)
print("=" * 78)
def section(name):
def wrap(fn):
def run(a, *kw):
banner(name)
try:
out = fn(a, *kw)
RESULTS[name] = out if isinstance(out, str) else "ok"
return out
except Exception as e:
RESULTS[name] = f"SKIPPED / FAILED -> {type(e).name}: {e}"
print(f"\n[!] {name} did not complete: {type(e).name}: {e}")
traceback.print_exc(limit=3)
return None
return run
return wrap
banner("0. Install nvidia-cudnn-frontend and locate libcudnn")
subprocess.run(
[sys.executable, "-m", "pip", "install", "-q", "nvidia-cudnn-frontend"],
check=True,
)
import torch
assert torch.cuda.is_available(), "No GPU. Runtime -> Change runtime type -> GPU."
torch.backends.cudnn.enabled = True
_ = torch.nn.functional.con...