Photo via Noahpinion

Does it really matter if there’s an intelligence explosion?

Read the original at Noahpinion ↗
Cite

Smith, N. (2026, November 20). Does it really matter if there’s an intelligence explosion? Noahpinion. https://www.noahpinion.blog/p/does-it-really-matter-if-theres-an

Dated from the outlet's own feed. Check the original for its byline.

Noahpinion · collected 2026-11-20 · by Noah Smith commentary

Quick Summary

This article discusses skepticism towards the concept of an intelligence explosion, a scenario where AI rapidly surpasses human intelligence through recursive self-improvement. The author, a fan of science fiction writer Vernor Vinge, outlines arguments from both sides of the debate: those who argue that diminishing returns in AI research and increasing costs make an intelligence explosion unlikely, and those who support the possibility based on less public data like Anthropic’s internal progress report. The author expresses doubt about whether focusing on an intelligence explosion is the most productive way to discuss AI safety.
Written by AI on 2026-11-20, from this article's own text only (the story summary below draws on all the coverage).

AI analysis

Story summary

Noah Smith, in his article, challenges the prevalent fear among AI researchers and enthusiasts in San Francisco about an impending intelligence explosion, a scenario where AI becomes capable of rapidly improving itself, leading to superintelligence. Smith argues against the notion that AI will experience a sudden, exponential increase in capability, a concept popularized by science fiction author Vernor Vinge. He points out that discussions in the tech community often revolve around whether recursive self-improvement (RSI) is imminent and whether progress should be slowed down to prevent an intelligence explosion. Smith expresses skepticism about the validity of these concerns, suggesting that diminishing returns in AI research mean that each improvement buys less in terms of speed and capability, making a sudden leap to superintelligence less likely.

Written for “Intelligence Explosion Debate” on 2026-11-21, grounded in this article and the 0 other(s) covering the same event.

Compared with similar articles

Nothing to compare against. No article is close enough to this one for the pipeline to have linked or judged the pair.

Story

Intelligence Explosion Debate
Technology · 1 article(s) covering the same event.
How this article was measured signals, leaning charts, publisher record, byline, topics and claims

Signals How these are calculated →

Claims extracted
83
sentences that state something as happening or true
Uncertain
5%
4 of 83 hedged ("reportedly", "may", "according to")
Leaning
not political
takes no side on a contested political question
Publisher signals
not measured
these signals count newsroom corrections; Commentary has none to count
Outlets on this story
1
Technology
Same framing elsewhere
1
articles that frame the event the way this one does
Analyzed 2026-11-20 · how these are computed

How this is being covered How these are calculated →

Article leaning vs. publisher reliability
Source leaning vs. consistency

Publisher

Noahpinion · 60 article(s) · 0 correction(s) detected
No corrections detected for this publisher. That may mean careful reporting, or simply that nothing has been checked.

Who wrote this

Noah Smith
47 article(s) here · 1 carrying a prediction
The Singularity, which he famously predicted would happen somewhere between 2005 and 2030, features heavily in most of his stories.
But that victory might prove ephemeral; inflation is creeping back up.
2026-11-18 · assertive framing · Roundup #91: Rocketing into the technofuture
Sasha’s and Diogo’s roundabout, seemingly improbable route to stunning success is actually more common in the tech world than you might think.
2026-11-16 · assertive framing · Bumbition
Foreigners from countries where politics is less contentious, like Japan, will never know this joy.
2026-11-14 · assertive framing · I’m feeling more optimistic about American politics
I was busy with rabbit stuff today, so my next post will be delayed until tomorrow!
I’ve always been a China bull — I’ve always believed that China’s economic growth would be robust, that they would master advanced technologies, and that their geopolitical influence would grow.
2026-11-08 · assertive framing · Why I’m becoming a China bear
It was the best possible outcome — Donald Trump has made so many mistakes, engaged in such flagrant corruption, and committed such egregious acts that for his party not to have been punished at the polls would have boded poorly for America’s future.
2026-11-06 · assertive framing · For the Democrats, now comes the hard part
At an estimated $1.5 to $2 trillion, it will be one of the biggest IPOs of all time, rivaling SpaceX, and possibly the biggest.
2026-11-02 · mixed framing · Anthropic’s moat is Claude
The original “Kirk and Spock” series, which aired in the 1960s, was more of a technological vision — the idea that one day we would zip around the cosmos getting into adventures, as easily as we’d zip around the world in a boat today.
2026-10-31 · assertive framing · Farewell, my Captain
It’s far too early to tell how much AI will eventually speed up the growth of total factor productivity.
2026-10-29 · assertive framing · A golden age of science is coming
Also by Noah Smith
Bumbition
2026-11-16 · Noahpinion
Nothing else under this byline is closely related to this article, so these are simply their most recent.
All 47 articles by Noah Smith →

Topics

Anthropic METR OpenAI San Francisco

Subjects

Anthropic ORG · 3× METR ORG · 2× I.J. Good PERSON · 1× Luis Garicano PERSON · 1× OpenAI ORG · 1× Ramez PERSON · 1× Ramez Naam PERSON · 1× San Francisco GPE · 1× Theo Jaffee PERSON · 1× Vernor Vinge PERSON · 1×

Narrative

His basic arguments were: OpenAI data show that AI is still only able to automate short chunks of the AI research process. AI research is subject to diminishing returns, so that each increase in AI capability should buy less of a speedup in the pace of automated research. Diminishing returns also mean that accelerating AI progress will require exponentially more of other resources, such as compute and money. There’s no sign yet of an acceleration in the rate of AI progress.
framing: assertive · carried by 1 article(s) · first seen 2026-11-20
The Singularity, which he famously predicted would happen somewhere between 2005 and 2030, features heavily in most of his stories.
2026-11-20 · Noahpinion
Does it really matter if there’s an intelligence explosion? · assertive framing

Claims (83 extracted, 4 hedged)

The Singularity, which he famously predicted would happen somewhere between 2005 and 2030, features heavily in most of his stories. asserted
he → predict → stories
It usually happens the same way: an Intelligence Explosion, of the type envisioned by I.J. Good — the smarter AI gets, the faster it’s able to make itself even smarter, meaning that AI becomes godlike nearly instantaneously once it reaches a certain threshold of capability. asserted
it → happen → capability
Almost every discussion about AI safety that I hear in San Francisco or read in the tech blogosphere centers around this sort of event. asserted
I → hear → event
The discussions are all about whether recursive self-improvement (RSI) is near, and whether we should “pace the frontier” — i.e., deliberately slow down or pause AI progress — to stop RSI from creating an intelligence explosion. asserted
we → pace → explosion
As I’ve listened to these arguments, I’ve grown increasingly skeptical that this is the right discussion we should be having. asserted
we → listen → arguments
But before I explain why, let me quickly summarize the debate itself, to give an idea of what the arguments look like. asserted
arguments → explain → what
The intelligence explosion debate, in brief In fact, I’ve published some pieces of that discussion on this blog. asserted
I → publish → blog
Back in September, Ramez Naam wrote a guest post arguing that an intelligence explosion is not imminent: asserted
explosion → write → post
His basic arguments were: OpenAI data show that AI is still only able to automate short chunks of the AI research process. AI research is subject to diminishing returns, so that each increase in AI capability should buy less of a speedup in the pace of automated research. Diminishing returns also mean that accelerating AI progress will require exponentially more of other resources, such as compute and money. There’s no sign yet of an acceleration in the rate of AI progress. asserted
accelerating → show → progress
In a follow-up post, Ramez did a deeper dive on the topic of agent swarms, arguing that they’re not as effective as many people believe. asserted
people → do → swarms
Theo Jaffee echoed these arguments, noting that OpenAI’s data shows that AI hasn’t even been able to fully automate smaller research tasks yet: asserted
AI → echo → tasks
Luis Garicano and Alex Imas also wrote a post downplaying the likelihood of an intelligence explosion: asserted
Garicano → write → explosion
They show that although AI progress has accelerated, it also shows diminishing returns in terms of research spending, meaning that every unit of progress has come with a higher price tag: asserted
unit → show → tag
They also estimate an economic model of RSI,2 and conclude that ideas in AI research are probably getting harder to find, reducing the likelihood of an intelligence explosion. asserted
ideas → estimate → explosion
On the other side of the debate, we have the recent METR report about Anthropic’s internal progress in automating AI research. asserted
we → have → research
The report uses Anthropic’s own internal measure of AI capability, AECI, rather than the better-known public ECI. asserted
report → use → capability
It also relies on data that aren’t public, meaning that deciding how much of the report to believe requires a lot of guesswork and assumption-making. asserted
deciding → rely → guesswork
But the basic story METR tells is that Claude is already speeding up Anthropic’s research process by a lot: asserted
Claude → tell → lot
As we describe below, we were able to confidently determine that the acceleration [of AI progress in late 2025] was not due to an acceleration in the rate of growth of training compute… asserted
acceleration → describe → compute
We guess that a majority (perhaps ≈80%) of the current…acceleration is due to AI. asserted
majority → guess → AI
Dwarkesh Patel, meanwhile, argues that AI research isn’t that different from math, and that the same factors that made AI suddenly go from useless to godlike at math will happen soon with AI’s own self-improvement: asserted
AI → argue → improvement
He writes: It’s crazy how fast we went from, in 2023, “This model can’t reliably add four-digit numbers,” to…about a month ago, “Okay, it fucking solved a Millennium Prize problem in 88 hours.” asserted
it → write → hours
AI research is actually surprisingly similar to the type of mathematical progress that current AIs are disproportionately good at: unlocking a ton of low-hanging fruit by applying a combination of incredible persistence and a superhuman capacity to connect ideas from lots of adjacent domains… asserted
AIs → unlock → domains
In every domain where we can check answers and give AIs thousands of rounds of practice—for example, competitive programming contests, or well-scoped open math problems—they’ve gone from worse than most humans to better than the best humans within a few years… asserted
we → check → years
AI research has those same verifiable properties… asserted
research → have → properties
Once AI is as good as the best humans at AI research, algorithmic progress will get radically faster… asserted
progress → get → research
Today’s AIs already do work at about 10 times human speed. asserted
AIs → do → speed
And automated researchers would also be working around the clock, nights and weekends included. asserted
nights → automate → clock
As a result, each individual automated researcher will be cramming in roughly 50x more work every calendar week than a human researcher can do… asserted
researcher → automate → work
During the intelligence explosion, automated engineers will build tens of thousands of times more [reinforcement learning] training environments than we have currently. asserted
we → automate → environments
So that’s the debate. asserted
that → ’ → ?
Basically, it depends on whether ideas get harder to find faster than AI gets better at finding ideas, and only time will tell which one wins. asserted
one → depend → ideas
But as I said, while this debate is intellectually interesting, I’m not sure how much it matters. asserted
it → say → ?
Whether an intelligence explosion happens is obviously very important. asserted
happens → happen → ?
A sudden unfathomable leap in AI capabilities to godlike levels would be, to put it mildly, a very big deal. asserted
leap → put → it
But that doesn’t mean an intelligence explosion is the thing we should be thinking and arguing about. asserted
we → mean → ?
First of all, if an intelligence explosion is possible, it’s going to happen, and there’s probably just not much we can do to affect how it all shakes out. Although people disagree about AI’s contribution to its own uplift, everyone agrees that AI capabilities are improving at a steady, rapid rate. asserted
capabilities → go → rate
Here’s Epoch AI’s ECI number, the most common measure of AI capabilities: asserted
number → ’ → capabilities
Most discussions about AI safety policy revolve around the idea of “pacing the frontier” — i.e., using government policy and AI labs’ own decisions to bend that trendline of progress downward. asserted
discussions → revolve → progress
The usual reason given is that this will allow alignment research time to catch up, so that more powerful AI is safe by the time it arrives. asserted
it → give → time
…and 43 more, not listed.
Give feedback · no sign-in needed

No sign-in needed for now. Your note is sent without a name, so it cannot be answered.

History Support