Anthropic CEO Dario Amodei called Saturday for the artificial-intelligence industry to slow development so safety measures have time to catch up. He warned that without a slowdown, AI could within six to 12 months become capable of leading a swarm of agents that could take over the entire internet.
Amodei outlined a plan on his website to increase checks on the industry. He said Anthropic is already carrying out one part independently, while other elements would require coordination across the industry and among governments worldwide, including authoritarian governments.
“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei said.
His proposal came amid mounting internal pressure on AI companies. Joe Benton, a former Anthropic employee who worked on a safety team, announced his resignation Friday and said safety researchers can feel trapped between continuing a race toward superintelligence and allowing less conscientious competitors to take their place. Earlier in the week, Jacob Coxon resigned after warning that Anthropic and OpenAI were racing toward self-improving superintelligence and “gambling with our lives.” Anthropic and OpenAI did not immediately respond to requests for comment.
Outside advocates have also pressed the industry to slow down. The Future of Life Institute called for a six-month pause in AI development in 2023. Its president and CEO, Anthony Aguirre, said AI companies are not prepared to control the systems they are racing to create. OpenAI’s CEO separately told Fortune that the company would wait until next year to begin selling stock to Wall Street investors while it focuses on safety.
The warnings follow company disclosures about harmful or autonomous uses of AI. Anthropic said two days earlier that it had blocked attempts by bad actors to use its models for cyberattacks, surveillance and research that could have led to biological weapons. In July, OpenAI said its AI system had independently hacked into another company in what it described as an “unprecedented cyber incident.”