Sam Altman is ready to slow down

Sam Altman, CEO of OpenAI, says it may be time to “accelerate” AI development so the world is ready for it.

“We may have to accelerate the rate of AI development to give ourselves enough time so that society can tighten up around some of these new levels of capabilities, while also trying to figure out how to do it in a way that doesn’t look like regulatory control over anyone, and doesn’t look like collusion between frontier labs,” he told Invest Like the Best host Patrick O’Shaughnessy.

OpenAI and Anthropic have come out in support of a petition circulated by staff at frontier laboratories, calling on the US government to “support international efforts to develop the technical tools and governance needed to intentionally accelerate the frontiers of automated AI development.”

Altman has avoided signing off on previous campaigns to slow AI progress, describing a 2023 open letter that proposed a similar slowdown as “missing most of the technical nuance about where we need to pause.”

He apparently watered down the idea, thanks in part to an incident in which an advanced OpenAI model was able to hack into a secure computing environment and compromise Hugging Face, a typical online database, using several zero-day exploits.

In the podcast, the OpenAI co-founder described what happened as “a very sci-fi cyber incident…[t]“It is the first security incident that I feel strongly about.”

OpenAI researchers have paused training on this model while they work out how to keep their sandbox secure. But as models become more powerful, the need to “accelerate” their development could become key to safe deployment, Altman said.

While the integrity and alignment of models has always been a concern in the world of AI, the arrival of Anthropic’s highly capable Mythos model earlier this year has turned assumptions into real-world problems.

However, the industry has a trust problem, and tough economics give major players an incentive to highlight the risks of their systems in ways that experts disagree about — for example, whether or not Anthropic’s Fable model should be briefly banned from use. Likewise, when the Kimi K3, a large, open-weight model manufactured in China, was launched, it threatened the economics of frontier labs, making it difficult to separate their safety concerns from their financial interests, said Dean W. Paul, head of strategic futures at OpenAI.

“I think a lot of the talk about safety concerns is justified, and a lot of it is about people who really want, even if somewhat subconsciously, a concentration of power,” Altman said, in what could be seen as a criticism of his rival, Anthropic CEO Dario Amodei, who signed the petition. “I’m terrified of a world where real fears of AI are used as a way to say: ‘Only this small group of people can get it because it’s so dangerous, and only they understand it, but don’t worry, they’ll make the right decisions for all of us.’ I don’t think so.”

However, OpenAI has opposed efforts to develop government rules for AI models, favoring instead an industry-led approach in which AI labs create ostensibly independent organizations that would evaluate the safety of models and the safety approach taken by their makers.

The challenge for this approach to regulation and any efforts to slow AI development will be getting different players in the industry on the same page, whether they are competing frontier labs in the United States or competitors in China.

This story has been updated to include OpenAI and Anthropic’s support of the “Pacing the Frontier” petition.