Why SoC Interconnects Have Outgrown the Bus

The shared bus was the backbone of chip design for decades. Today, it is the bottleneck.

As system-on-chip (SoC) architectures evolve through 2025 and into 2026, the assumptions that made the bus an elegant, universal solution no longer hold. Modern SoCs pack dozens of heterogeneous compute engines, wide memory subsystems, and increasingly chiplets connected through advanced packaging. The simple, arbitrated bus cannot keep pace—and the industry has responded by rethinking the on-die interconnect from the ground up.

The Bus: A Solution That Outlived Its Assumptions

The classical bus is a shared communication medium. Every master—CPU, GPU, DMA engine, or accelerator—competes for the same wires, governed by an arbiter. This model is simple, well-understood, and cheap to implement. For the homogeneous, single-core or few-core designs of the 1990s and 2000s, it was ideal.

The bus carried two implicit assumptions:

A limited number of masters. Contention stays manageable when only a handful of blocks want to talk.
Uniform, modest bandwidth demand. Traffic is roughly comparable across blocks, and aggregate throughput fits within the shared medium.

Both assumptions have collapsed.

What Changed

Explosion in Master Count

A modern AI-capable SoC may contain hundreds of initiator and target blocks: multiple CPU clusters, several GPU or NPU arrays, DSPs, image signal processors, video codecs, high-speed I/O controllers, and dozens of sensor and peripheral interfaces. A single shared bus serializes all of this traffic, and arbitration overhead grows with every added master.

Bandwidth-Hungry Accelerators

Generative AI and large-model inference have pushed memory and interconnect bandwidth to the top of the design agenda. Accelerators now demand hundreds of gigabytes per second and beyond, far exceeding what a shared bus can deliver. By 2026, on-die bandwidth—not raw compute—is often the limiting factor in AI SoC performance.

Heterogeneous, Non-Uniform Traffic

Traffic in modern SoCs is bursty and asymmetric. A neural accelerator may demand sustained peak bandwidth, while a management controller needs only occasional, latency-sensitive transactions. A shared bus forces these very different flows to compete on equal terms, degrading quality of service for the applications that need it most.

The Chiplet Era

Advanced packaging and chiplet-based designs—standardized around interfaces such as UCIe—have turned the SoC into a system of systems. Interconnect must now span dies, not just functional blocks, and must handle coherency, protocol translation, and physical distance that the on-die bus was never designed for.

The Shift to Network-on-Chip and Beyond

The replacement for the bus is the network-on-chip (NoC). Rather than a single shared medium, a NoC routes packets between nodes through a topology of switches and links, much like a scaled-down data-center network.

Key advantages include:

Scalability. Adding nodes adds bandwidth rather than contention.
Parallelism. Multiple transactions proceed simultaneously over distinct paths.
Quality of service. Traffic classes, priorities, and virtual channels let latency-sensitive flows be isolated from bulk transfers.
Physical design flexibility. NoCs decouple logical connectivity from physical placement, easing timing closure and floorplanning.

Building on this foundation, 2025–2026 designs increasingly layer in:

Coherent fabrics that extend cache coherency across CPUs, accelerators, and chiplets.
Memory-centric interconnect with near-memory and in-memory compute nodes to reduce data movement.
Optical and die-to-die links for high-bandwidth, low-energy chiplet communication.

Implications for Designers

The bus era taught a generation of engineers to think in terms of arbitration and shared resources. The NoC era demands a different mindset:

Treat interconnect as a first-class architectural decision, not an afterthought bolted on late.
Model traffic and QoS early, because interconnect choices constrain achievable bandwidth and latency.
Plan for heterogeneity, including mixed protocols, coherency domains, and chiplet boundaries.
Co-design with physical implementation, since topology and placement now strongly interact.

The bus is not disappearing entirely—simple peripherals still use it. But for the high-performance core of a 2026 SoC, the shared bus has been decisively outgrown.

The Bottom Line

SoC interconnects have outgrown the bus because the bus was built for a simpler world: few masters, uniform traffic, and modest bandwidth. Modern designs—driven by AI acceleration, heterogeneous compute, and chiplet-based packaging—require the scalability, parallelism, and quality of service that only a networked interconnect can provide. The NoC, and its evolving descendants, is now the backbone of the SoC.

via Semiconductor Engineering

Why SoC Interconnects Have Outgrown the Bus