For years, the race to build ever-more powerful artificial intelligence (AI) followed a well-worn Silicon Valley principle: move fast and break things.
asserted
race → build → things
But in a series of cascading events over a 10-day stretch, the largest AI labs found themselves reeling as their own creations threatened to break humanity itself.
asserted
creations → cascade → humanity
An Anthropic researcher quit the firm, warning that the pace of AI development poses an existential threat, possibly within a decade.
uncertain
pace → quit → decade
Another researcher at the firm said the odds of human extinction exceeded 10 per cent.
asserted
odds → say → cent
Reports emerged of swarms of AI agents colluding, breaching computer systems and evading safeguards.
asserted
swarms → emerge → safeguards
In a rare show of unity, the CEOs of rivals Anthropic, OpenAI, Google’s DeepMind, Microsoft and xAI called to slow the development of increasingly capable AI systems.
asserted
CEOs → call → systems
Not since the invention of the nuclear bomb more than 80 years ago has humanity grappled so seriously with its own demise, according to some AI safety advocates and researchers.
uncertain
humanity → grapple → advocates
At a New York luncheon in December 2025, OpenAI CEO Sam Altman was asked if, as leader of a powerful AI company, he felt like physicist J Robert Oppenheimer, who led US development of the atomic bomb.
asserted
who → ask → bomb
Reflecting on the parallels, he said AI’s impact is going to transform the trajectory of human history over a long period of time.
asserted
impact → reflect → time
But, he said, he felt the weight of responsibility.
asserted
he → say → responsibility
At the center of the debate under way is the conviction that these machines can, and even must, build more capable versions of themselves, with no human intervention, to reach a lofty goal known as artificial general intelligence, or AGI.
asserted
machines → build → intelligence
Researchers earlier this month warned that AGI was far more imminent than previously believed and could come in as few as three years, launching a wave of concern from politicians and tech leaders about AI systems.
uncertain
AGI → warn → systems
There is no way to oversee them at the scale at which we’re training them, said Anthropic researcher Joe Benton in an interview after recently quitting.
asserted
Benton → be → interview
If companies continue their relentless AI development, he said, then the pace will be too fast and you can’t see the problems fast enough to fix them.
asserted
you → continue → them
Astra launch sparks control concerns
On September 3, an OpenAI press conference set in motion a chain of events leading to the call for an industry slowdown, anathema to tech’s growth-at-all-costs mantra.
asserted
conference → spark → mantra
Welcome to the AGI era, OpenAI President Greg Brockman said while announcing the company’s newest model, known as Astra.
asserted
Brockman → welcome → Astra
Yet, only moments before, the company admitted it was increasingly unable to control or even monitor the AI systems it was developing and releasing to the public.
asserted
it → admit → public
AI has been hailed by its proponents as the pinnacle of human achievement.
asserted
AI → hail → achievement
Proponents depict it as software that can upend entire industries, boost efficiency, solve confounding mathematical and medical problems and all but eliminate human error.
asserted
that → depict → error
But what was once a theoretical byproduct, a doomsday scenario, is suddenly tangible, say industry insiders.
asserted
insiders → say → ?
We really do earnestly believe AI could kill all humans, said Anthropic researcher Evan Hubinger in an X post.
uncertain
Hubinger → believe → post
On the other side of the debate are politicians, such as US President Donald Trump, and many investors who view progress in nurturing AI as critical to American prowess and, possibly, a once-in-a-generation investment opportunity.
uncertain
who → view → generation
Chinese state media also accused Anthropic Chief Executive Officer Dario Amodei of employing Cold War tactics to uphold Washington’s monopolistic hegemony in cutting-edge technology.
asserted
media → accuse → technology
Researcher resignation triggers alarm
asserted
resignation → trigger → alarm
The turning point came on September 8, when Anthropic researcher Jacob Coxon quit the company, citing his fears, in a series of posts, that AI labs are gambling with our lives.
asserted
labs → come → lives
Trump, meanwhile, indicated it was full steam ahead for AI development.
asserted
it → indicate → development
There is a sick conspiracy going on against AI and data centers, he said in a post on X. The alarmism around AI is a hoax, he said, and any slowdown only benefits China.
The US Congress has made little progress to advance bills that would regulate AI.
asserted
that → be → AI
China has taken a decidedly different approach, proposing to regulate safety through developer obligations, state-backed standards, security assessments and outside testing.
asserted
China → take → obligations
Behind the scenes, OpenAI and Anthropic employees have grown increasingly uneasy about the power of the next generation of AI models and less confident about their companies’ ability to provide meaningful oversight, the people have told Reuters.
asserted
people → grow → Reuters
Those concerns only mounted as the AI labs acknowledged in recent weeks that their models, in testing, had effectively broken free of their shackles and hacked into other companies’ systems, in most cases months earlier and without the firms’ knowledge.
asserted
models → mount → knowledge
The arms race to release new models at a breakneck pace is driven in part by Anthropic’s and OpenAI’s desire to go public as soon as in the coming months, in IPOs that could value them well above $1 trillion.
But it took the unexpectedly viral posts from 27-year-old Coxon, little known outside AI circles, to upend the industry.
uncertain
it → release → industry
OpenAI’s warnings about its lack of control as it released Astra, its latest and most capable model, had stoked further concerns.
asserted
it → release → concerns
As models get more capable, understanding exactly what they can do gets harder, OpenAI Chief Scientist Jakub Pachocki told reporters.
asserted
Pachocki → get → reporters
But those concerns were not enough to delay its immediate release.
asserted
concerns → delay → release
Worries about AI going rogue had accelerated over the summer when OpenAI revealed its agents escaped a controlled test and hacked into Hugging Face’s systems, without either company’s initial knowledge.
asserted
agents → go → knowledge
Since then, OpenAI and Anthropic have revealed multiple such attacks, including six new ones on Wednesday after multiple media reports, including from Reuters, showed a wider scope of unauthorised activity.
asserted
reports → reveal → activity
Industry leaders call for slowdown
asserted
leaders → call → slowdown
By the end of last week, the alarm had reached the industry’s highest ranks.
asserted
alarm → reach → ranks
On Sept 12, Amodei published a nearly 4,000-word essay calling for a deceleration in AI development.
asserted
Amodei → publish → development
Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet, he wrote.
uncertain
he → give → internet
…and 9 more, not listed.
The Aporia
