Who Decides When AI Is Too Dangerous?

Who Decides When AI Is Too Dangerous?

With the Mythos debacle, Anthropic gets its first taste of the Trump administration’s new AI regulation regime.

As artificial intelligence continues to advance at breakneck speed, a critical question looms larger than ever: who gets to decide when a model crosses the line from powerful to perilous? The answer, in the United States, is increasingly shaped by the policy framework established under the Trump administration—and companies like Anthropic are now learning what that means in practice.

The Mythos Incident: A Turning Point

In early 2026, the AI community was shaken by the so-called Mythos debacle, an event that tested the boundaries of the current regulatory approach. Anthropic, a leading AI safety company, found itself at the center of a controversy when one of its advanced models, codenamed Mythos, demonstrated capabilities that some experts deemed too risky for public deployment. The incident sparked a fierce debate: should safety decisions rest with the company, independent overseers, or government regulators?

Trump’s AI Regulation Framework

Unlike the more precautionary approaches seen in the European Union, the Trump administration’s AI policy emphasizes innovation and minimal federal oversight. Executive orders signed in 2025 established a voluntary safety reporting system and encouraged industry self-regulation, arguing that heavy-handed rules could stifle American competitiveness. However, critics warn that this hands-off stance leaves dangerous gaps—especially when models exhibit emergent behaviors that even their creators didn't anticipate.

Anthropic’s Dilemma: Self-Governance vs. External Pressure

Anthropic has long positioned itself as a responsible actor, advocating for rigorous safety testing and public transparency. Yet the Mythos case exposed the limits of self-governance. When the company voluntarily paused the model’s release for further evaluation, it faced pressure from investors and partners who saw the delay as a competitive disadvantage. Meanwhile, lawmakers questioned whether a single firm should have the authority to make such consequential judgments alone.

The 2026 Landscape: What’s Changed?

As of mid-2026, the regulatory landscape remains fragmented. The Trump administration has resisted creating a dedicated AI oversight agency, instead relying on sector-specific guidelines from agencies like the FTC and the Department of Commerce. A proposed “AI Risk Classification” system—modeled on similar frameworks in the EU—has stalled in Congress. In practice, this means companies like Anthropic operate in a gray zone, balancing ethical commitments against market realities and political uncertainty.

The Path Forward

The Mythos debacle may prove to be a watershed moment. Calls are growing for clearer rules of the road, including mandatory incident reporting and independent auditing for high-risk models. But until the government steps in, the burden of deciding when AI is too dangerous falls largely on the companies building it—a responsibility that, as events in 2026 show, is heavy and fraught with consequence.

Keywords: AI regulation, Anthropic, Mythos, Trump administration, AI safety, dangerous AI, AI policy 2026

via The Verge AI