Researchers warn of AI extinction risk after resignations

Researchers from Anthropic and OpenAI warn of a 10%+ risk of AI-driven human extinction within ten years, criticizing the reckless race toward superintelligence.

Jacob Coxon, a researcher formerly involved in training AI models at OpenAI and Anthropic, has resigned, expressing grave concerns about the unchecked race toward self-improving superintelligence. He accuses both companies of acting irresponsibly, likening their actions to 'playing Russian roulette with our lives,' and warns that AI could pose a threat to humanity by the end of the decade. Other Anthropic researchers, including Jan Leike and Evan Hubinger, estimate the risk of human extinction from AI at over 10% within the next ten years. These warnings underscore a significant credibility gap in AI governance and safety practices. The situation has sparked calls for improved coordination between AI labs, stronger independent audits, and a slowdown in deploying advanced models. Mounting concerns may affect company valuations and drive regulators and clients to demand greater oversight and transparency.