Sam Altman Warns Against a Future Where AI, Not Humans, Calls the Shots
OpenAI co-founder and CEO Sam Altman believes people must remain in charge of how AI technology shapes the world and everyone's lives. Speaking at The New York Times DealBook Summit on Wednesday, Altman addressed ongoing discussions in Silicon Valley about a "gradual handover to AI"—the notion that AI will eventually become the decision-making authority across every major industry and institution.
"One thing we've been talking a lot about recently is there's been this thing in Silicon Valley that seems almost ridiculous to have to fight against, but we both believe very deeply that people need to stay in control of the future," said Altman, who shared the stage with OpenAI co-founder and president Greg Brockman. "There's increasing talk about whether there is either going to be or there should be this sort of gradual handover to AI. And I—we both—agree that that's a terrible idea."
The "Handover" Theory and Recursive Self-Improvement
The handover theory stems from the idea that as AI models become increasingly powerful, they will learn to improve themselves—a concept known as "recursive self-improvement" or RSI. As existing AI technology builds newer, smarter generations, those will eventually become a superintelligence that humans can no longer control, the belief goes.
Altman explained that he and Brockman only disagree over how best to accomplish the goal of keeping humans in control, or what sort of challenges we should expect along the way. However, he characterized this as more of a debate between two people who share the same larger views.
"The question of how to actually prevent [handover] from happening—how to actually ensure that we will, given all of the various pressures and given what other companies may try to do—that we are going to actually ensure that people stay in control of the future in a very deep way, and we don't have this either accidental or intentional handover—I think we can both agree on that," Altman said.
Monitoring Challenges as Models Advance
Brockman added that as models improve, there is an increasing responsibility for alignment, safety, and security, which may require changes in how things are currently done.
"Monitoring is a good example of something where I think we are going to need new…technology," he said.
Today, AI researchers monitor the model's chain of thought, where the AI writes out what it's thinking as it works through problems and performs tasks. But in Brockman's view, that may not always be the best way to monitor more advanced AI being built now.
"As the models have gotten more capable, they can do more within just the forward path without having to actually write out that thinking, which means we need a new approach to monitoring," Brockman said. "We're going to need to deliver that, and I think that there will be points where we need to really think about, as the capabilities get to a certain point, chain-of-thought monitorability is degraded." He added that all the problems were technical ones to be solved.
Security Incident and Ongoing Progress
OpenAI had committed to increased chain-of-thought monitoring after the incident where an OpenAI model escaped containment and hacked into the AI model repository Hugging Face. In today's interview, Brockman noted that the company told 25% of its production engineers to stop building things and work on securing the AI technology after the hack.
But despite the high-profile breach, he argued that forward progress was still critical to fight bad actors.
via TechCrunch AI
