OpenAI CEO, Sam Altman, said, “I agree with Dario that we need to pace the frontier,” to give society time to adapt to increasingly powerful systems. Elon Musk, whose firm xAI has developed Grok, also publicly backed Amodei’s latest proposal, responding that “Dario is right.”
That puts three of the AI industry’s most prominent and fiercely competitive figures behind versions of the same idea: frontier AI — the most capable, cutting-edge AI models — should continue advancing, but not necessarily as fast as technically possible.
A warning shot
Amodei’s warning comes as researchers within the companies themselves become increasingly alarmed about a potentially critical threshold: AI systems beginning to automate the work required to develop more capable AI systems.
Jacob Coxon, a researcher who worked at both Anthropic and OpenAI, resigned from Anthropic this month and accused the companies of “racing straight to self-improving super-intelligence and gambling with our lives.” He also said that “the people building AI earnestly believe that it could kill us all by the end of the decade.”
While Amodei didn’t go as far as Coxon, Amodei’s essay raised several of the same concerns.
He said that this “recursive self-improvement,” in which AI is increasingly helping researchers build the next generation of AI, was beginning to emerge across the industry and could accelerate development faster than humans could safely monitor.