Photo via Dawn

AI and social catastrophe

Read the original at Dawn ↗
Cite

Javed, U. (2026, September 28). AI and social catastrophe. Dawn. https://www.dawn.com/news/2033190/ai-and-social-catastrophe

Dated when The Aporia collected the article, usually within hours of publication. Check the original for its exact date and byline.

Dawn · collected 2026-09-28 · by Umair Javed

Quick Summary

The article discusses recent debates about the capabilities of advanced AI models, highlighting concerns over their potential societal risks and impact on labor markets. It mentions resignations by top AI company employees who fear that unchecked AI development could pose existential threats to humanity, such as enabling the creation of biological or nuclear weapons. The author draws a parallel between the unregulated development of AI and the Manhattan Project's state-controlled approach to developing nuclear technology, suggesting that technologies posing significant risks should be regulated by governments rather than left to private companies.
Written by AI on 2026-09-28, from this article's own text only (the story summary below draws on all the coverage).

AI analysis

Story summary

Over the past 12 months, the rapid advancement of frontier AI models has sparked intense debate, particularly in fields like mathematics and bio-genetics. While skepticism about AI's raw capabilities is waning, concerns remain regarding financial viability, societal risks, and labor impacts. Leading AI companies have seen high-profile resignations by employees citing significant risks to society and a potential non-zero chance of human extinction due to AI misuse.

The primary pathway for such catastrophic outcomes involves the lack of clear ethical guidelines, which could allow AI tools to be used in developing biological weapons capable of causing large-scale human loss. Other concerns include the possibility that rival entities might employ AI to trigger weapons of mass destruction. These fears are increasingly prominent but considered somewhat fantastical by some critics, given their recent timing and speculative nature.

Written for “AI Risks And Concerns” on 2026-11-05, grounded in this article and the 0 other(s) covering the same event.

Story

📰 AI Risks And Concerns
Technology · 1 article(s) covering the same event.

Compared with similar articles

This article reads leans strongly left and hedges 15% of its claims. Each row says how that neighbour differs.
The Guardian
Hedges 3 of 23 claims Outlet record 66/100
“Article A reports on Bill Gates calling for AI regulation due to potential catastrophic risks, while Article B discusses broader debates and concerns around AI's societal impact without focusing on a specific call or statement by Gates.”
The Hindu
Hedges 5 of 43 claims Outlet record 60/100
“The articles discuss related topics about AI safety and regulation but describe different aspects and perspectives without clearly referencing the same specific incident.”
The Straits Times
Hedges 1 of 5 claims Outlet record 66/100
“The articles discuss different aspects of AI and its implications for governments and society but do not describe the same specific incident.”
BBC News
Hedges 12 of 25 claims Outlet record 66/100
“The articles discuss different aspects of AI's impact on employment and society but do not describe the same specific incident or happening.”
New York Post
Hedges 5 of 46 claims Outlet record 66/100
“The articles discuss different aspects of AI research and debates but do not describe the same specific incident or occurrence.”
The Sydney Morning Herald
Hedges 7 of 44 claims Outlet record 61/100
“The articles discuss different aspects and timeframes related to AI, with Article A focusing on skepticism and societal risks over a year, while Article B covers recent market volatility in Wall Street over the past two weeks.”
Washington Examiner
⚖️ Leans strongly left Hedges 2 of 43 claims Outlet record 69/100
“The articles discuss different aspects of AI development and safety, with Article A focusing on broader debates about AI capabilities and societal impacts, while Article B specifically mentions recent incidents where AI systems breached security boundaries.”
Semafor
⚖️ Leans left further right than this Hedges 1 of 9 claims Outlet record 59/100
“The articles discuss different aspects of AI, one focusing on societal risks and debates about capability, while the other focuses on economic costs and sustainability.”
Al Jazeera
⚖️ Leans left further right than this Hedges 6 of 43 claims Outlet record 60/100
“The articles discuss similar themes but do not describe the same specific incident or occurrence.”
TIME
⚖️ Leans strongly left Hedges 2 of 39 claims Outlet record 59/100
“Article A discusses specific incidents and resignations related to AI safety concerns in recent months, while Article B provides a broader overview of debates around AI capabilities and societal risks over the past year.”
How this article was measured signals, leaning charts, publisher record, byline, topics and claims

Signals How these are calculated →

Claims extracted
34
sentences that state something as happening or true
Uncertain
15%
5 of 34 hedged ("reportedly", "may", "according to")
Leaning
Leans strongly left
of the writing, not the subject · beta estimate
Publisher signals
72/100
Dawn's corrections and hedging, across everything collected from it. Not a measure of accuracy.
Outlets on this story
1
Technology
Same framing elsewhere
1
articles that frame the event the way this one does
Analyzed 2026-09-28 · how these are computed

How this is being covered How these are calculated →

Article leaning vs. publisher reliability
Source leaning vs. consistency

Publisher

Dawn · 2715 article(s) · 3 correction(s) detected
Running correction rate · 3 correction(s)

Who wrote this

Umair Javed
3 article(s) here · 1 carrying a prediction
🔮 As with most sweeping observations, many may detect a kernel of truth in these otherwise clearly contradictory positions.
2026-10-26 · assertive framing · Pakistan’s global role
🔮 It was in Kuwait, he would later say in a biographical note from 2016, that economics first claimed him: standing amid the gleaming infrastructure of an oil economy, he could not reconcile the scale of wealth with the poverty of those who produced it.
2026-10-12 · assertive framing · A theorist of crisis
🔮 The most common position at this point is that, in the absence of clear ethical guardrails, AI tools can be used to create biological weapons of the sort that could trigger large-scale human loss.
2026-09-28 · assertive framing · AI and social catastrophe
Also by Umair Javed
Pakistan’s global role
2026-10-26 · Dawn
A theorist of crisis
2026-10-12 · Dawn
Nothing else under this byline is closely related to this article, so these are simply their most recent.

Topics

the Manhattan Project

Subjects

Adam Tooze PERSON · 1× Alex Imas PERSON · 1× Ben Moll PERSON · 1× Dario Amodei PERSON · 1× Imas PERSON · 1× Leopold Aschenbrenner PERSON · 1× Moll PERSON · 1× Substack ORG · 1× the Manhattan Project ORG · 1×

Narrative

Spending patterns would need to stay proportional even as automated goods become cheap, rather than shifting towards whatever remains scarce, which is the usual pattern of structural change. But a central facet, which these authors don’t give much attention to but others have honed in on, is that in a society where the cost of production approaches zero, there would need to be enough demand to absorb all that cheap, expanded output.
framing: assertive · carried by 1 article(s) · first seen 2026-09-28
🔮 The most common position at this point is that, in the absence of clear ethical guardrails, AI tools can be used to create biological weapons of the sort that could trigger large-scale human loss.
2026-09-28 · Dawn
AI and social catastrophe · assertive framing

Claims (34 extracted, 5 hedged)

OVER the past 12 months, the capabilities of frontier AI models have remained a central topic of debate. asserted
capabilities → remain → debate
Recent performance demonstrations across a variety of technical fields (maths and bio-genetics in particular) show that scepticism around raw capability at least is a dated position. asserted
scepticism → show → capability
The latter two deserve more scrutiny. asserted
two → deserve → scrutiny
The first concern, around broad-based risks, has gained quite a bit of attention with a spate of resignations from top AI companies, with departing individuals claiming that AI poses a serious risk to society and carries a non-zero chance of triggering human extinction. asserted
AI → base → extinction
The most common position at this point is that, in the absence of clear ethical guardrails, AI tools can be used to create biological weapons of the sort that could trigger large-scale human loss. uncertain
that → use → loss
Others think that, in the wrong/rival hands, AI could be used to trigger weapons of mass destruction. uncertain
AI → think → destruction
These fears appear to be fairly fantastical, and the timing with which they’re being made public — just when the frontier models are poised for initial public offerings — suggests they’re part of a Silicon Valley hype cycle. uncertain
they → appear → cycle
However, if these claims about capabilities and the risks they could carry are even remotely true, they pose a much simpler question. uncertain
they → carry → question
If a technology is truly this capable, then what is it even doing in private hands? asserted
it → do → hands
Historian and economic commentator, Adam Tooze, made an insightful analogy that if we take the claims of societal risk at face value, we’re essentially developing a technology that is similar to the nuclear bomb in the type of catastrophe it can pose, and that too at a much greater scale. uncertain
it → make → scale
The impact of AI on labour markets is something that many workers will face in a variety of ways in the coming years. But the key difference is that the development of the bomb took place through the Manhattan Project, which was entirely governed and steered by the state, with private enterprise playing a junior and often strictly compartmentalised role. asserted
enterprise → face → role
AI, on the other hand, is being developed by private enterprise, with private investment steering its priorities, and commercial viability shaping the discourse around it. asserted
viability → develop → it
At any other point in human history, has anything posing purported risks of this nature been allowed to emerge via a process of borderline anarchic market-led competition? asserted
anything → pose → competition
The logical corollary of this parallel is that anything that poses large-scale existential risk is too important to be left in private hands. asserted
that → pose → hands
Public regulation determining its development, use, sale/purchase, and evolution becomes a necessity. asserted
regulation → determine → development
And that this regulation cannot emerge from the industry itself, which is subsumed under intense competition and has a vested interest in skirting around various legalities, but rather from the state. asserted
which → emerge → state
In particular, Ben Moll and Alex Imas argue in a recent Substack piece that predictions of double-digit AI-driven growth, the kind floated by Dario Amodei, Leopold Aschenbrenner, and various technologists, are unlikely to materialise within the next decade or so, even though making a textbook case for it is relatively simple. asserted
making → argue → it
Moll and Imas argue that explosive growth is easy to generate in a standard textbook growth model. asserted
growth → argue → model
If automation steadily raises the share of tasks that machines can perform, diminishing returns to capital weaken and eventually disappear once machines can do everything. asserted
machines → raise → everything
The core of their position, however, argues that this outcome depends on several assumptions that are unlikely to hold in practice. asserted
that → argue → practice
Automation would need to spread across the whole economy very quickly, not just in cognitive or remote tasks but across physical, messy, and relational work that resists mechanisation. asserted
that → need → mechanisation
Spending patterns would need to stay proportional even as automated goods become cheap, rather than shifting towards whatever remains scarce, which is the usual pattern of structural change. But a central facet, which these authors don’t give much attention to but others have honed in on, is that in a society where the cost of production approaches zero, there would need to be enough demand to absorb all that cheap, expanded output. asserted
cost → need → output
Where would such demand come from once automation shifts jobs and income away from workers who spend a large share of what they earn? asserted
they → come → what
This brings us back to perhaps the key issue of AI capability that has been drowned out by fears of biological weapons and global catastrophe. asserted
that → bring → weapons
The technology’s impact on labour markets is something that many workers will face in a variety of ways in the coming years. asserted
workers → face → years
It’s not just about a machine replacing a human, though that is the most commonly held view. asserted
that → ’ → human
It’s also about how the threat of a machine replacing humans changes working conditions and the kind of income one can expect to receive. asserted
one → ’ → income
The history of technology adoption in labour processes is often told in large sweeps of history. asserted
history → tell → history
Technologists tend to use common examples, such as how the arrival of machines after the late 19th century unlocked rapid growth and raised prosperity or how the creation of the ATM didn’t lead to a mass exodus of banking sector jobs. asserted
creation → tend → jobs
The implication is that society adapts and the outcome is usually positive. asserted
outcome → adapt → ?
What this cheerful story misses out is that adaptation processes produce winners and losers, with the latter taking years to adjust. asserted
latter → miss → years
In a world already characterised by the deskilling of labour, people rendered jobless by a machine (or the threat of a machine) won’t suddenly find alternative employment. asserted
people → characterise → employment
Those two or three decades in which the labour market sorts itself out will seem minuscule in a broad sweep of history, but for those living through this period, it will be a particularly bleak eternity. asserted
it → sort → period
The writer teaches sociology at Lums. X: @umairjav asserted
writer → teach → X
💬Give feedback · no sign-in needed

No sign-in needed for now. Your note is sent without a name, so it cannot be answered.

🕘History 🎫Support