Anthropic Proposes AI Safety Bill That Could Block Model Releases if 50+ People Are at Risk of Dying
Over the past year, concerns have mounted over the potential dangers posed by artificial intelligence, and debates have intensified over the right path for AI safety. Now, Anthropic appears to have put forth its own vision for how the government might regulate its industry.
A Draft Bill for Congress
Politico recently reported that the AI lab has been circulating a new draft bill on AI safety that it hopes Congress might consider adopting. The legislation outlines certain scenarios in which the release of an AI model could be blocked from distribution.
What Counts as Catastrophic Risk
Those scenarios involve a "foreseeable, imminent risk" β one in which more than 50 people are at risk of dying, or if more than a billion dollars in property damage could result, according to Politico. The catastrophic risk in question must "stem from the creation or release of a biological weapon, the model evading its developer's controls or a cyberattack," the outlet notes.
The metrics offered here β 50 people and a billion dollars β are similar to those in a piece of California legislation recently passed into law, Senate Bill 53, which likewise defined those scenarios as tantamount to "catastrophic risk."
Independent Evaluators and a 72-Hour Clock
Anthropic's bill suggests that analysis of a model's potential for disaster could be conducted by "independent evaluators" registered with the U.S. Commerce Department. These evaluators could potentially be paid by the AI labs themselves β so long as the "payment is not contingent on the evaluations' results," Politico writes.
It is not clear how this proposed system would ensure that payments could not influence the evaluation. Demis Hassabis, chairman of DeepMind, posited some months ago that a self-governing AI regulatory body should mimic the financial industry's Financial Industry Regulatory Authority (FINRA).
Under Anthropic's draft bill, when evaluators determine that an imminent catastrophic risk is posed, they would have 72 hours to notify the Commerce Department. The Commerce Department would then have to request that a D.C. judge halt the model's release. The legislation would expressly forbid any federal agency from having the discretion to halt the model's distribution on its own, Politico writes.
Anthropic Confirms the Draft
Anthropic did not immediately respond to TechCrunch's request for comment, but a spokesperson did confirm the draft bill to Politico. "This bill represents the proposals outlined in Anthropic's Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June," the spokesperson said.
The Risks vs. the Headlines
The theoretical risks outlined in the draft policy are somewhat removed from the kinds of incidents that have been top of mind for the tech industry of late. While there are people warning of the potential for human death, the primary concern has largely been the threat of agentic hacking. A rash of incidents in which AI programs have apparently disobeyed their human operators and hacked websites on the open internet has spurred concern that models may be lacking proper security controls.
In an effort to present a solution to the spectacle of crisis that the labs themselves have helped bring about, companies like Anthropic have offered up their own preferred solutions.
via TechCrunch AI