OpenAI's Chief Scientist Proposes a Way to Limit AI Development

OpenAI's Chief Scientist Proposes a Way to Limit AI Development


A new blog post from OpenAI's chief scientist, Jakub Pachocki, discusses potential limits on AI development, with a focus on a measurement called perplexity. The post arrives amid intensifying debate about how fast frontier AI should advance.


The Debate Over Pacing AI


Slowing AI development became a particularly hot topic after a former OpenAI and Anthropic researcher said the companies were "gambling with our lives," and Anthropic's IPO filing even suggested that the technology could pose "catastrophic or existential risks to society."


Anthropic CEO Dario Amodei then published a plan to "pace the frontier," seemingly endorsed by OpenAI CEO Sam Altman and SpaceX CEO Elon Musk. Among other things, Amodei called for companies to agree to "common safety standards as well as limits on the rate of unchecked AI progress."


Why Perplexity?


But to establish a speed limit on AI development, you need an agreed-upon measurement for speed. Pachocki's post, described by Altman as "the most important thing OpenAI has published in a while," suggests perplexity (not to be confused with the company of the same name) could be that measurement.


In Pachocki's argument, benchmarks that focus on AI's "downstream capabilities" have quickly become outdated as the technology advances. Perplexity, however, has remained "informative." It "measures how well a model predicts the next token on average; lower perplexity means better predictions."


Pachocki's post includes a chart of how the perplexity of OpenAI's models has fallen since GPT-3.5.


"GPT-4, at ~0.91, was proficient at utilizing chain-of-thought reasoning productively, which motivated the subsequent targeted elicitation of this capability and the creation of modern reasoning models," he wrote. "Now, as we have reached losses below 0.8, we have seen the emergence of transformative capabilities in fields like cybersecurity and mathematics."


A Shared Benchmark and Common Limits


In his view, perplexity could provide the basis for "a shared benchmark for general intelligence," which AI companies could then use to establish "common limits." With perplexity serving as a standard measurement for AI progress, the companies would commit not to go past a certain threshold "until our ability to understand and monitor AI behavior achieves a certain concrete milestone."


The proposal comes as the industry looks for measurable, comparable yardsticks to underpin the safety commitments that frontier labs have increasingly pledged in 2026.

via TechCrunch AI

Related