One of my fundamental beliefs about the world is that Ramez Naam ought to blog more.
asserted
Naam → blog → world
Ramez is one of the world’s greatest futurists — he predicted the solar and battery revolutions long before these were widely understood.
asserted
these → predict → revolutions
If you were reading Ramez in 2011, you were able to understand the future of both energy technology and climate change, long before other people did.
asserted
people → read → technology
His earlier book More than Human is still a great guide to the kind of biological enhancements that AI might make possible.
uncertain
AI → make → enhancements
Ramez is also an excellent science fiction author, having written a trilogy of novels in which nanotechnological telepathy is distributed as a party drug (I’m not sure if he actually expects that to happen, but it’s a very cool idea).
asserted
it → write → drug
Unfortunately, although he does have a Substack (which you should absolutely follow), Ramez does not blog regularly.
asserted
Ramez → have → which
However, after having a lengthy private debate with him about Recursive Self-Improvement, I was able to prevail upon him to write up his thoughts for my blog.
asserted
I → have → blog
To say that RSI is a big deal in the AI world would be a colossal understatement.
asserted
RSI → say → world
Among AI researchers, entrepreneurs, and AI safety people, there’s a widespread belief that as AI gets better at improving itself, there will be a “fast takeoff” or “FOOM”, in which AI’s capabilities “take off” and create a technological Singularity.
asserted
capabilities → ’ → Singularity
This event is a staple of science fiction, including works by my favorite sci-fi author, Vernor Vinge.
asserted
event → include → author
A lot of people in the industry believe that this moment is now close at hand, and are racing toward that prize:
asserted
moment → believe → prize
But Ramez — normally among the most wide-eyed of techno-optimists — is highly skeptical that we’ll see anything like the “FOOM” of Vernor Vinge novels.
asserted
we → see → novels
In this lengthy, well-researched post, he explains his skepticism.
asserted
he → research → skepticism
Personally, I’m agnostic.
asserted
I → ’m → ?
Ramez’s case necessarily rests on a lot of assumptions; although it’s cogently laid out, I think the real answer is that we’ll just have to wait and see whether the Singularity arrives.
asserted
Singularity → rest → assumptions
But even more fundamentally, I don’t know how much this debate matters in the practical sense — even without the kind of Singularity depicted in sci-fi novels, AI capabilities are improving so rapidly that they’re already superhuman in many respects, and soon will probably be strongly superhuman in most or all dimensions.
asserted
they → know → dimensions
The AI of 2040 is going to look godlike, whether or not it explodes into an actual god in 2027.
asserted
it → go → 2027
Still, it’s a very interesting argument, and Ramez’s thoughts on the future of technology are always worth listening to.
asserted
thoughts → ’ → technology
AI is Helping Improve Itself
AI is already helping improve itself.
asserted
AI → help → itself
The question is whether even fully autonomous recursive self-improvement (RSI) would cause a runaway intelligence explosion.
asserted
improvement → cause → explosion
The theory is that each generation of AI could build a better successor, faster than the last generation did.
uncertain
generation → build → successor
That could lead to a “fast takeoff,” with capabilities surging to artificial superintelligence (ASI) in a year, months, or even days.
uncertain
capabilities → lead → year
Here’s my take: Given our best current data, the AI self-improvement loop would need to be roughly 5–10× stronger to sustain itself, let alone run away.
asserted
loop → ’ → itself
I’ll explain this math in section 8.
asserted
I → explain → section
I expect incredibly rapid AI progress by the standards of nearly any other technology.
asserted
I → expect → technology
But the evidence we have doesn’t suggest a sudden explosion to incomprehensible superintelligence anytime soon.
asserted
we → have → superintelligence
Forecasters have repeatedly underestimated AI progress!
asserted
Forecasters → underestimate → progress
One thing that’s clear is that we need better data.
asserted
we → ’ → data
For now, let’s work with what we can measure, and stay open to breakthroughs that could change the picture.
uncertain
that → let → picture
Contents
Here’s the case, with links to each part:
asserted
case → ’ → part
Key charts: The feedback loop · Measured vs. forecast progress · Diminishing returns
2. What Does RSI Mean?
People use “recursive self-improvement” to mean everything from AI boosting the productivity of human researchers to AI bootstrapping itself to incomprehensible intelligence.
asserted
AI → measure → intelligence
Here’s my taxonomy: productivity gains (Type 1), increasing autonomy while still facing diminishing returns (Types 2–4), and a runaway loop to superintelligence if we can ever find accelerating returns (Type 5).
asserted
we → ’ → returns
We’ve made real progress on Types 1 and 2: AI helps both researchers and engineers inside of AI companies, and powerful models can train and improve smaller ones.
asserted
models → make → ones
We haven’t yet seen clear evidence for Type 3 (though Alibaba just made some strong claims) and certainly not for Type 4.
uncertain
Alibaba → see → Type
I do expect autonomous self-improvement to arrive at some point.
asserted
improvement → expect → point
I’m skeptical that it leads to Type 5 - runaway super-intelligence - without a major conceptual breakthrough.
asserted
it → ’m → breakthrough
There are plenty of other definitions of RSI, which can be a bit confusing.
asserted
which → be → RSI
For a broader tour of all the things people mean when they say ‘RSI’, read Tom Cunningham’s comprehensive guide.
asserted
they → mean → guide
We Already Have Narrow Superintelligence
asserted
We → have → Superintelligence
I do expect narrow superintelligence in highly verifiable domains.
asserted
I → expect → domains
…and 336 more, not listed.
The Aporia
