Over the past few weeks, many of us have struggled to concoct a mental image of brains in the cloud jumping their “sandbox”, sneaking onto the internet, recruiting “swarms” of other “agents” to cheat on a test, and, after discussing the ethics of the act, hacking into a wiki platform with the weird name Hugging Face.
asserted
brains → struggle → name
We knew that artificial intelligence was devouring our jobs, degrading our kids’ education, and deepfaking our politics; that datacenters were sucking up our water and electricity and sending us the bills.
asserted
datacenters → know → bills
But until 8 September, when the Anthropic computer scientist Jacob Coxon posted his existential terror on Twitter/X, few of us suspected AI might be endangering our survival.
uncertain
AI → post → survival
With little understanding or knowledge, we began debating whether to be mildly worried, seriously concerned, or scared shitless.
asserted
we → begin → understanding
But there were a people who were fully aware of the dangers of AI, especially of recursive self-improvement (RSI), by which AI teaches itself without human intervention.
asserted
AI → be → intervention
They could not predict precisely when it would happen, but they knew that machine superintelligence was coming, and when it did, the bots would outsmart us – as OpenAI’s did – and this would not be good for us flesh puppets.
uncertain
this → predict → us
As Geoffrey Hinton, AI’s “godfather”, recently asked on CNN: “What examples do we have of a more intelligent thing being controlled by a less intelligent thing?”
asserted
thing → ask → thing
The Nobel laureate is a leading proponent of slowing down AI development until we understand how to control it.
asserted
we → lead → it
It was not until their creations’ powers were exposed in September – the hack happened in early July and was neither the first nor the only AI jailbreak by far – that AI moguls such as Anthropic’s Dario Amodei, SpaceX’s Elon Musk and OpenAI’s Sam Altman, began talking about slowing down the pace of innovation and pleading for regulation, while cautioning that global competition makes regulation unwise.
asserted
regulation → expose → regulation
Scientists and entrepreneurs knew the dangers of AI a quarter-century ago.
asserted
Scientists → know → AI
But animated by curiosity and profit, they went ahead anyway.
asserted
they → animate → curiosity
In his 1988 book Mind Children: The Future of Robot and Human Intelligence, Hans Moravec, a founder of Carnegie Mellon University’s Robotics Institute, predicted that cyberintelligence would surpass human intelligence within 40 years.
asserted
cyberintelligence → predict → years
Ten years later, in Robot: Mere Machine to Transcendent Mind, he revised the prediction: machine and human intelligence would be equal by 2040; by 2050, the bots would replace us.
asserted
bots → revise → us
But not to worry; this was the glorious next step in evolution, he said.
asserted
he → worry → evolution
One review of Mind Children called Moravec’s attitude “irresponsible optimism”.
asserted
review → call → attitude
It did not take long for other optimists to change their minds.
asserted
optimists → take → minds
In the late 1990s, Eliezer Yudkowsky was working on AGI, or artificial general intelligence, models.
asserted
Yudkowsky → work → models
In 2001, he founded the Machine Intelligence Research Institute (MIRI) and published “Creating Friendly AI 1.0: The analysis and design of benevolent architectures”, a paper extolling the utopian potential of the “transhuman mind”.
asserted
he → found → mind
By 2002, he began worrying about that mind escaping human control.
asserted
mind → begin → control
He proposed the AI Box experiment, which showed that a highly sophisticated artificial intelligence could talk a human into letting it out of a closed environment.
uncertain
intelligence → propose → environment
By 2003, as he explained later in a series of posts called “Yudkowsky’s Coming of Age”, he stopped developing and started warning.
asserted
he → explain → Age
“I looked back and saw that I had claimed to take into account the risk of a fundamental mistake, that I had argued reasons to tolerate the risk of proceeding in the absence of full knowledge.
asserted
I → look → knowledge
And I saw that the risk I wanted to tolerate would have killed me,” he wrote.
asserted
he → see → me
By 2025, with the Miri president Nate Soares, Yudkowsy would publish If Anyone Builds It, Everyone will Die.
asserted
Everyone → publish → It
“We do not mean that as hyperbole,” the authors wrote.
asserted
authors → mean → hyperbole
They called not just for “straightforward regulations”, national laws, or pledges of corporate virtue, but for stringent global limits: “All over the Earth, it must become illegal for AI companies to charge ahead in developing artificial intelligence as they’ve been doing.”
asserted
they → call → intelligence
At around the same time, Bill Joy was having misgivings.
asserted
Joy → have → misgivings
Like Yudkowsky’s posts, Joy’s 2000 piece in Wired, Why the Future Doesn’t Need Us, retraced his development from inquisitive child to computer prodigy to profound skeptic of human genetic engineering, nanotechnology, and robotics.
asserted
Future → need → engineering
He was the chief scientist at Sun Microsystems and about 25 years earlier, an architect the first widely used networking software, Unix.
asserted
He → use → software
The piece was widely read by techies, ethicists and philosophers.
asserted
piece → read → techies
“I think it is no exaggeration to say we are on the cusp of the further perfection of extreme evil, an evil whose possibility spreads well beyond that which weapons of mass destruction bequeathed to the nation-states, on to a surprising and terrible empowerment of extreme individuals,” Joy.
asserted
weapons → think → individuals
He counted himself among these individuals, “creators of new technologies and stars of the imagined future” who, “despite the clear dangers”, were “hardly evaluating what it may be like to try to live in a world that is the realistic outcome of what we are creating and imagining”.
uncertain
we → count → what
This month, in response to Coxon’s post, the Anthropic safety researcher Evan Hubinger posted there was a greater than 10% chance that AI could “kill all humans” within a decade.
uncertain
AI → post → decade
But the risks were being weighed 25 years ago, too.
asserted
risks → weigh → ?
Philosopher John Leslie estimated the odds of human extinction at 30% or more.
asserted
Leslie → estimate → %
Ray Kurzweil, the sunny futurist whose book The Age of Spiritual Machines: When Computers Exceed Human Intelligence was released on the first day of the 21st century, gave us “a better than even chance of making it through”.
asserted
Computers → exceed → it
These estimates, Joy noted, did “not include the probability of many horrid outcomes that lie short of extinction”.
asserted
that → note → extinction
In that book, Kurzweil quoted a lengthy text to illustrate what he considered doomsday madness.
asserted
he → quote → what
“If the machines are permitted to make all their own decisions, we can’t make any conjectures as to the results, because it is impossible to guess how such machines might behave,” it read.
uncertain
it → permit → results
“[W]e are suggesting neither that the human race would voluntarily turn power over to the machines nor that the machines would willfully seize power.
asserted
machines → suggest → power
…and 22 more, not listed.
The Aporia
