A debate over whether artificial intelligence could have something resembling consciousness has put Anthropic, one of the world's leading AI companies, at odds with the Vatican after Pope Leo XIV rejected the idea that machines could replace or surpass human beings.
uncertain
machines → resemble → beings
The disagreement surfaced after Anthropic co-founder Christopher Olah reviewed the Pope's first encyclical, "Magnifica Humanitas", ahead of its launch at the Vatican.
asserted
Olah → surface → Vatican
Olah had accepted an invitation to speak at the event on May 25, but reportedly considered pulling Anthropic out after disagreeing with the Pope's position on AI consciousness.
uncertain
Olah → accept → consciousness
The dispute points to a much bigger question surrounding the rapid development of AI.
asserted
dispute → point → AI
As companies build increasingly capable systems and deploy them in everyday life, there is still disagreement over whether these systems should be understood simply as tools, autonomous agents or something closer to beings.
asserted
systems → build → beings
What did Pope Leo XIV say about AI
In his encyclical, Leo warned against allowing technology to become more important than human beings.
asserted
technology → say → beings
" ... humanity — in all its grandeur and woundedness — must never be replaced or surpassed"
asserted
humanity → replace → grandeur
He also wrote, "If technology becomes the ultimate criterion, the human person risks being reduced to data, a cog in a machine or a commodity."
asserted
person → write → machine
The Pope's position clashes with questions Olah has been raising privately about whether advanced AI systems could possess some form of moral status.
uncertain
systems → clash → status
According to The New York Times, Olah has encouraged religious thinkers to consider the possibility that Anthropic's Claude models could have something resembling moral status.
uncertain
models → accord → status
Why is Anthropic thinking about AI's moral status
Anthropic has spent months consulting religious scholars and other experts about how AI systems should behave and what values they should follow.
asserted
they → think → values
The company has held private discussions with scholars from around the world and required participants to sign nondisclosure agreements, according to the Times.
uncertain
company → hold → Times
The effort comes as Anthropic pursues rapid expansion, with the company potentially reaching a valuation of $2 trillion.
asserted
company → come → trillion
Olah has argued that AI development cannot be separated from questions about values and morality.
asserted
development → argue → values
"Those who control AI will impose their own moral vision, which will become the invisible infrastructure of these systems," Olah argued, per the Times.
asserted
Olah → control → Times
His concerns go beyond philosophical questions.
asserted
concerns → go → questions
Anthropic has been examining possible real-world risks from AI, including disruption to jobs, cyberattacks and biological threats.
asserted
Anthropic → examine → jobs
AI systems are already being introduced into workplaces, customer services, search, education and other areas where their decisions can affect people's lives.
asserted
decisions → introduce → lives
Anthropic's attempt to decide what AI should value
Anthropic says it is trying to address these concerns through outside consultation and a written "constitution" for Claude.
asserted
it → decide → Claude
The 84-page document is intended to guide how the AI model behaves.
asserted
model → intend → ?
The company has also consulted philosophers, religious scholars, psychologists and civil society representatives.
asserted
company → consult → philosophers
Anthropic's in-house philosopher Amanda Askell said she wants the company's models to be "the best of us".
asserted
models → say → us
Olah, meanwhile, has argued for "some shared notion of goodness that cuts across society in some very broad way," according to the Times.
uncertain
that → argue → Times
But questions remain over who should ultimately decide what values powerful AI systems should follow.
asserted
systems → remain → values
Private discussions between technology companies and outside experts may influence AI behaviour, but they do not amount to public oversight.
uncertain
they → influence → oversight
Olah has also said researchers are encountering behaviour in AI systems that they do not fully understand.
asserted
they → say → that
"I will be honest: We keep finding things that are mysterious, even unsettling.
asserted
that → keep → things
We find structures that mirror results from human neuroscience.
asserted
that → find → neuroscience
We find evidence of introspection.
asserted
We → find → introspection
We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease.
asserted
that → find → joy
I don't know what that means, but I think it warrants ongoing discernment," Olah said, according to the Times.
uncertain
Olah → know → Times
At the centre of the dispute is therefore a question that goes beyond the Vatican and Anthropic.
asserted
that → go → Vatican
As AI becomes more capable and increasingly embedded in society, researchers, technology companies and religious thinkers are still debating what these systems actually are, what rights or moral status they might have, and who should decide the values that guide them.
uncertain
that → become → them
The Aporia
