We’ve been hearing a great deal recently about the specter of AI-driven human extinction.
asserted
We → hear → extinction
Maybe our misplaced trust in AI intelligence will set off World War III.
asserted
trust → misplace → III
Or AI agents will unleash widespread cyberattacks and take over the internet.
asserted
agents → unleash → internet
Given too much autonomy, the machines will run amok.
asserted
machines → give → autonomy
They may seek to deceive their creators—wait, that’s already happened.
uncertain
that → seek → creators
Suppose they conclude, like Agent Smith in The Matrix, that “human beings…are a plague, and we are the cure.”
asserted
we → suppose → Matrix
Or perhaps the chatbots help bad humans unleash a literal plague—a biological weapon.
asserted
humans → help → plague
A “misuse” report Anthropic made public earlier this month detailed five cases in which foreign scientists tried to evade Claude’s guardrails on “gain of function” inquiries.
asserted
scientists → make → inquiries
That’s a euphemism for when scientists tinker with a pathogen’s genes to enhance it in some way, making it more infectious, deadlier, or better able to spread among mammals, for example.
asserted
it → ’ → example
Anthropic denied those users the answers they sought and cut off their accounts, even while noting that their motives may not have been sinister.
uncertain
motives → deny → accounts
Such information also can be used to help scientist develop pandemic countermeasures like vaccines and drugs.
asserted
scientist → use → vaccines
Like so many technologies, AI included, genetic engineering can be used for both good and evil.
asserted
engineering → include → good
I studied to be a molecular biologist, and have been covering the gain of function debate since long before AI was on everyone’s radar.
asserted
AI → study → radar
In my experience, researchers who work on high-risk pathogens tend to default to the view that it’s better to publish than not, and that we’re better off doing the work to understand which genetic traits make organisms a threat to humankind—never mind the potentially catastrophic consequences of an accidental release, or a deliberate one.
asserted
organisms → work → release
Is humanity really better off doing the work?
asserted
humanity → do → work
I have my doubts.
asserted
I → have → doubts
And I have more of them after reading the latest book from Annie Jacobsen, whom I last interviewed about her best-selling book Nuclear War: A Scenario.
asserted
I → have → book
Her new book, Biological War: A Scenario, came out in late July and follows the same template.
asserted
book → come → template
And because I know you were about to ask, yes, I’m sure she is great fun at cocktail parties.
asserted
she → know → parties
One thing about Jacobsen, who has decades of high-level national security reporting under her belt, is her unrivaled access to current and former military and government officials and scientists—people who spend their days thinking about threats to humanity and gaming out how the government might respond if such-and-such horrific thing were to happen.
uncertain
thing → have → humanity
Beyond the notion of human extinction, perhaps the most terrifying aspect of both of her books relates to the limitations in our government’s preparation and response.
asserted
aspect → relate → preparation
America’s general plan for nuclear war was, and no doubt remains, complete madness.
asserted
plan → remain → war
As for biological war, spoiler alert, it turns out we really can’t defend against it.
asserted
we → turn → it
(You’re welcome.)
asserted
You → ’re → ?
I’ve edited our conversation for length and clarity.
asserted
I → edit → length
So, Annie, now that you’ve done both the nuclear scenario and the biological scenario, which is more likely to keep you up at night?
asserted
which → do → night
The Defense Department document that I reprinted in the book, on the weapons of mass destruction threat spectrum, lists nuclear weapons at the far end of the curve because they are the most destructive.
asserted
they → reprint → curve
But I was really surprised to see how biological weapons are a close second.
asserted
weapons → see → ?
And I believe the reason they’re the greatest overall threat very much ties into the discussion the entire world is having today, which is that the threshold for entry is extremely low, and therefore it is more likely to happen.
asserted
it → believe → entry
To have a nuclear weapon, you have to have weapons-grade fissile material, and then you have to think about containing those weapons within the arsenal.
asserted
you → have → arsenal
You have to have an ICBM, a B-2 bomber, a nuclear-armed submarine.
asserted
You → have → ICBM
You’ve got to be a state!
asserted
You → get → ?
A terrorist with a dirty bomb, that’s just a completely different threat.
asserted
that → ’ → bomb
The component that makes biological war so incredibly dangerous and the greatest overall threat is that there are nine nuclear-armed states tasked with this containment component, but then you look at the rest of the world and who has access to [potential pandemic agents]…
Most everyone.
asserted
who → make → everyone
And that becomes terrifying.
asserted
that → become → ?
If you were a bioweapons designer, what would you look for in a pathogen?
asserted
you → look → pathogen
My mind does not think that way because I am a mother and I’m a member of society.
asserted
I → think → society
But If I were a pandemic planner, what would scare me the most is the concept of an airborne agent and the public’s naiveté to that very concept.
asserted
scare → scare → concept
I would think about how important it is to educate the public about the reality of what is an extraordinary threat and what isn’t.
asserted
what → think → reality
Your scenario involves the accidental release of a weaponized pathogen from a high security lab in Russia.
asserted
scenario → involve → Russia
…and 113 more, not listed.
The Aporia
