Teams operating coding agents inevitably face a common bottleneck. Claude Code relies on the Anthropic Messages API, Codex CLI uses OpenAI's interface, yet the model the team intends to deploy may reside behind vLLM, NVIDIA NIM, or Ollama. Since rewriting the agent is impractical, the translation layer must live elsewhere.
Switchyard is NVIDIA's answer: a Rust-based proxy and library for LLM traffic that routes requests across providers, translates between OpenAI and Anthropic formats, records operational metrics, and offers typed, composable routing algorithms. Released under the Apache 2.0 license, its documentation is available at docs.nvidia.com/nemo/switchyard. As of 2026, Switchyard remains an experimental project, though it has gained traction among developers seeking a flexible routing layer for hybrid LLM deployments.
Is it deployable? Yes, but only for evaluation purposes. The binary can be installed from crates.io, and a launcher is available from PyPI. It self-hosts on any infrastructure, yet NVIDIA labels Switchyard as pre-alpha and experimental, explicitly warning against production use and noting that the API and algorithms are likely to change substantially before version 1.0.
What Switchyard Does
Switchyard ensures that clients retain their native API. The software decodes inbound requests into provider-neutral Rust types, executes a routing algorithm to select a backend, re-encodes the request into that backend's wire format, calls the backend, and translates the response—including streaming events—back into the format the original client expects.
The server component accepts three inbound connection types: OpenAI-compatible endpoints, Anthropic-compatible endpoints, and AWS Bedrock. This flexibility allows organizations to standardize on one internal interface while benefiting from multiple upstream providers, avoiding vendor lock-in without altering agent code.
Routing and Observability
A key feature is its support for dynamic routing strategies. Administrators can define rules based on model availability, latency, cost, or custom heuristics, all expressed through a typed Rust interface. Beyond routing, Switchyard logs detailed operational metrics—such as request counts, latency percentiles, and token usage—providing visibility into multi-provider traffic flows that is often absent in single-vendor setups.
Limitations and Roadmap
Given its pre-alpha status, Switchyard is not recommended for mission-critical workloads. The project's maintainers anticipate breaking changes and are actively seeking community feedback to shape the v1.0 release. Early adopters should expect to invest time in understanding the architecture and adapting to evolving APIs. Production hardening—including comprehensive error handling, security audits, and performance optimization—remains on the roadmap.
via MarkTechPost