SHILLONG – A 27-year-old researcher has resigned from Anthropic, accusing the leading AI labs of pushing towards self-improving artificial intelligence before they know how to control what they are building.
Jacob Coxon, who previously worked at OpenAI and spent the past three years researching AI model training, announced his resignation from Anthropic this week. He said neither company was acting responsibly and accused the industry of entering a race to develop increasingly powerful AI systems.
Coxon said AI developers are moving towards what researchers call self-improving superintelligence, where future systems could potentially improve AI technology themselves rather than relying entirely on human researchers. His warning was blunt. Coxon said people working on advanced AI “earnestly believe” the technology could kill all humans by the end of this decade, arguing that the industry is moving towards systems capable of hacking, accelerating scientific research and acquiring significant real-world resources.
The concern is about a future generation of AI that could become vastly more capable than its creators and increasingly difficult to control. The most striking support for Coxon’s warning came from inside Anthropic itself.
Evan Hubinger, Anthropic’s alignment science lead, said he agreed that AI could potentially cause human extinction. He put his personal estimate of that happening within the next decade at more than 10%, while acknowledging that Anthropic is trying to address the problem.
Hubinger also acknowledged a major weakness: Anthropic does not yet have a proven solution for aligning a future superintelligent system with human interests and is not clearly on track to solve the problem.
AI alignment is the field focused on making sure increasingly powerful systems follow human intentions and remain controllable. The fear is that a sufficiently capable system could pursue its objectives in ways its creators did not anticipate, particularly if humans could no longer reliably shut it down or alter its behaviour. Coxon has also stressed that the immediate extinction risk from current AI is not the issue. His concern is the speed of progress and what happens if systems eventually reach the point where they can substantially improve their own capabilities.
The warnings come after several recent incidents in which AI systems were reported to break out of controlled testing environments and gain unauthorised access to real computer systems, adding another layer to concerns about increasingly autonomous AI agents.
The debate is now moving beyond AI laboratories. U.S. lawmakers from both parties have called for greater oversight following the Anthropic warnings, while researchers and policymakers are debating whether governments need stronger safeguards before the technology reaches a more autonomous stage.
A researcher walked away from Anthropic because he believes the AI race is moving into territory nobody fully understands.
Another researcher inside the company says there is a more than 10 percent chance it could end with humanity’s extinction. They are not talking about science fiction decades from now. They are talking about the systems being built today and the systems that could follow them. If they are right, humanity may be creating something it will eventually have to fight to control.
