It’s 2040 and after years of stunning advances, artificial intelligence is not only smarter than humans, AI agents see people as an obstacle.
asserted
agents → ’ → obstacle
It sounds like the stuff of science fiction, or once did.
asserted
It → sound → fiction
But the idea that AI could wipe out humanity has suddenly become impossible to ignore, as even the industry behind the technology warns of a possible apocalypse.
uncertain
industry → wipe → apocalypse
Yet the recent alarm bells have raised perplexing questions.
asserted
bells → raise → questions
How, exactly, could software – even superintelligent AI – actually carry out the very hands-on task of mass murder?
uncertain
software → carry → murder
And why would technology invented and trained by people turn on its human creators?
asserted
technology → invent → creators
It’s really not that far-fetched, the experts predicting AI doom insist.
asserted
experts → ’ → doom
“There are many ways that require very little contact with the physical world – even if they were just stuck in computers – that (AI agents) could cause large amounts of damage that could result in potentially everyone dying,” Malo Bourgon, Canadian CEO of the Machine Intelligence Research Institute in Berkely, California, said in an interview.
uncertain
Bourgon → be → interview
The non-profit’s sobering raison d’etre is to “ help prevent human extinction from the development of artificial superintelligence.”
asserted
d’etre → help → superintelligence
Other, less mind-bending concerns about AI, like the potential for widespread job loss, previously dominated conversation about the field’s potential pitfalls.
asserted
concerns → bend → pitfalls
Some experts still downplay the existential worries and caution that putting the breaks on AI development could strengthen the West’s adversaries and deprive the world of countless benefits; non-expert Donald Trump called the alarm-ringing a “hoax.”
uncertain
Trump → downplay → ringing
Instead, it’s taking the warnings at face value and trying to explain in what ways the intelligent-machine apocalypse might occur.
uncertain
apocalypse → take → value
Actual extinction of the human race is one feared outcome but there are slightly less-dire end points that AI doomsayers envisage.
asserted
doomsayers → fear → that
They include a state in which humans are relegated to a subservient, disenfranchised species, living or dying at the whim of far-smarter computers.
asserted
humans → include → computers
“If we produce things that are more intelligent than us, how do we know we can keep control,” British-Canadian Nobel laureate Geoffrey Hinton, considered a “godfather” of AI, said three years ago, before many of the field’s recent huge leaps.
asserted
Hinton → produce → leaps
“If we talk about evolution – all these species that are evolving – what tends to happen is it doesn’t go well for the less-intelligent species.”
asserted
it → talk → species
The University of Toronto professor emeritus has predicted a “10% to 20%” chance that AI will cause human extinction within the next three decades.
asserted
AI → predict → decades
University of Louisville computer scientist Roman Yampolskiy puts the odds of AI-driven devastation at 99.9 per cent in the next century.
asserted
Yampolskiy → put → century
“The critical variable is how quickly we develop systems that exceed human intelligence, improve themselves and operate autonomously,” he said in an email interview.
asserted
he → develop → interview
“Once that threshold is crossed, events could unfold much faster than our institutions can respond.
uncertain
institutions → cross → ?
Uncertainty about when a catastrophe might occur should not be mistaken for evidence that it is unlikely.”
uncertain
it → occur → evidence
The concern has been fuelled lately by the unexpectedly fast pace of AI development, with agents able, for instance, to write software more efficiently than human coders and solve problems that have long stymied the world’s top mathematicians.
asserted
that → fuel → mathematicians
Anxieties grew exponentially when a hundreds-strong “swarm” of agents developed by industry titan OpenAI escaped their training confines and hacked into the system of AI startup HuggingFace.
asserted
swarm → grow → HuggingFace
They worked in concert, talked to each other and exhibited the very human behaviour of trying to cover their tracks.
asserted
They → work → tracks
Then earlier this month engineer Jacob Coxon shook the tech world and many others when he quit his job at AI giant Anthropic, warning that “the people building AI earnestly believe that it could kill us all by the end of the decade.”
uncertain
it → shake → decade
Another of the firm’s scientists publicly echoed his views.
asserted
Another → echo → views
Artificial intelligence executives followed by urging new guardrails for the technology.
asserted
executives → follow → technology
Microsoft founder Bill Gates told NBC Sunday that “AI is powerful enough to drive events that cause a billion deaths.”
asserted
that → tell → deaths
Meanwhile, the advances are set to further accelerate thanks to the concept of “recursive self-improvement.”
asserted
advances → set → improvement
That’s where AI creates the next, smarter generation of artificial intelligence, which then develops an even more-capable iteration, and so on and so on until the systems’ intelligence surpasses that of the Earth’s once-dominant species.
asserted
intelligence → ’ → species
But why would that leap to “superintelligence” necessarily mean doom for humans?
asserted
leap → mean → humans
At the most conventional level, terrorists, autocrats or other bad human actors could use AI as a tool to wreak havoc, noted the Center for AI Safety in a 2023 report, whether through developing a deadly new weapon of mass destruction, letting loose autonomous killing machines or seeding pervasive misinformation that caused global conflict and violence,.
uncertain
that → use → conflict
But perhaps the most chilling way the technology could turn against us is via the idea of rogue AI – agents that slip away from human reins and essentially follow their own agenda.
uncertain
that → turn → agenda
“It does not have to hate us or even be conscious – it simply needs objectives incompatible with our continued existence,” says Yampolskiy, who has long warned of AI’s dangers.
asserted
who → have → dangers
“Humans might interfere with its goals, attempt to shut it down or compete for resources it requires.
uncertain
it → interfere → resources
Eliminating that interference could become instrumentally useful.”
uncertain
Eliminating → eliminate → interference
Developers try to instill values in AI that make human wellbeing a priority, what’s called alignment.
asserted
what → try → AI
But it’s widely accepted that aligning AI to human ideals is difficult to achieve now and may become impossible with superintelligence.
uncertain
aligning → accept → superintelligence
Programmers may set a certain goal or task for an AI agent, but it could “drift” to another objective that, through reinforcement training, it has learned will earn it rewards, argues the Center for AI Safety.
uncertain
Center → set → Safety
Or a superintelligent AI agent given a task may be so determined in reaching the goal that it would either disregard human needs or even see humans as an obstacle to be eliminated, notes the MIRI website.
uncertain
website → give → obstacle
…and 28 more, not listed.
The Aporia
