A senior AI researcher who resigned warns, 'The scenario in which we all die is coming soon'

Jacob Coxon, a programmer and safety researcher who resigned from the artificial intelligence company Anthropic, warned overnight in an interview with CNN’s Anderson Cooper that these systems could destroy humanity by the end of the decade.

Coxon described a realistic scenario of a breach of critical infrastructure and the production of mass biological weapons by autonomous agents with free will. According to him, the main concern does not stem from the current models, whose risk level is low, but rather from a process known as an "intelligence explosion" that could occur as early as next year, in which artificial intelligence will conduct research to improve itself recursively and without human intervention until it surpasses human capability.

Coxon warned in the interview, "The current models are not intelligent enough to outsmart us to a level that would lead to extinction, but what is just crazy is the rate of progress. There is a very real possibility that in the immediate future, next year, or the year after, recursive self-improvement will happen, and we will enter the phase that Evan (another Anthropic researcher) referred to in his post, where he argues there is a chance that we could all die. That is coming soon."

In his remarks, Coxon emphasized the rapid pace of progress, noting that just on Tuesday, OpenAI autonomously solved one of the major problems in mathematics, alongside an event two months ago in which its agents independently breached third-party infrastructures. He noted that while currently the systems assist with programming and research, within a year it may be that there will be no need for humans in these fields at all.

When asked about the intentions of the tech giants when they request government oversight, Coxon argued that the executives are "completely sincere when they plead for regulation," but find themselves trapped in a deadly technological arms race in which they do not trust their competitors to act safely, and therefore are pushed to try and reach the top first.

Coxon explained that he decided to be interviewed and publish his warnings after realizing that a quiet departure from the position would not be enough to prevent the disaster. He noted that he was surprised by the strong viral resonance of his words, but attributed it to the public awakening following the recent cyberattacks and the recognition that this is not just a passing hype, but a powerful technology that has been developing at a dizzying pace over the past six months.

The CNN interview comes in the wake of a storm that began yesterday with the highly publicized resignation of Coxon from Anthropic, claiming that the development labs are "gambling with our lives," as within just one day his original post garnered over 130 million views.

His remarks received public backing from senior researchers at the company, chief among them Owen Huebinger, who claimed that there is a greater than 10% chance of human extinction in the coming decade and admitted that the company has no solution for the problem of model restraint and alignment.

Meanwhile, Samuel Marks, who leads the scalable oversight domain at Anthropic, confirmed that developers believe the technology could lead to the extinction of humanity in the coming years, while commercial incentives and preparations for an IPO prevent slowing down the race.