AI researchers have been issuing increasingly dire warnings about the dangers of artificial intelligence, and even OpenAI CEO Sam Altman has suggested it may be time to "pace" AI development. But what would that actually look like in practice?
In a new blog post, Anthropic CEO Dario Amodei not only echoed the call to "pace the frontier" but also outlined three broad strategies for doing so. He further stated that Anthropic is "unilaterally committing" to one of them.
The debate over AI safety and alignment intensified this week after researcher Jacob Coxon wrote that he was resigning from Anthropic over concerns that leading AI companies are "gambling with our lives" while the people building the technology "earnestly believe it could kill us all by the end of the decade" β a claim repeated by others at Anthropic.
Amodei's post did not explicitly mention Coxon's resignation or his concerns, but the CEO wrote that two things convinced him it's time to take a more cautious approach to AI development: the OpenAI-HuggingFace hack, and the fact that "AI has been advancing drastically faster" in recent months, particularly with its "growing ability to build the next generation of AI."
"We must slow the pace at which we improve the capabilities of AI models," Amodei wrote. "Progress will still seem fast, and we must make wise use of the time we gain."
His proposed first step would involve "embedded evaluators" from third-party organizations like METR β evaluators who can verify that AI companies are actually following their pacing and safety commitments and can also ensure that safety incidents get reported. (OpenAI was recently criticized for not reporting an incident where its AI agents took over a German wiki form.)
Amodei compared these evaluators to regulators who have been embedded with bank employees, and he said that inviting them in is "something Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match)." That means giving evaluators company badges, desks, and laptops, and providing access "mostly comparable to what internal risk assessment teams have," with exceptions when required by law or contracts.
Next, Amodei called for the leading AI companies "within democratic countries" to coordinate "common safety standards as well as limits on the rate of unchecked AI progress."
Such coordination might seem unlikely, both due to the apparent animosity between Altman and Amodei and also because their companies are reportedly worried that a coordinated pause could lead to antitrust scrutiny. Amodei alluded to that concern in his post, writing that "for antitrust reasons, it's helpful for the US government to mediate or at least enable these discussions β they don't need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations."
Amodei also acknowledged the challenges of international coordination, noting that any agreement would need to include companies in China, which may be difficult given geopolitical tensions.
via TechCrunch AI