A mere five years before breaking the record for the biggest public stock offering ever, Anthropic began as a Silicon Valley startup when its founders broke away from OpenAI with their own ideas about how to develop artificial intelligence.
asserted
founders → break → intelligence
Today, the company that dedicated itself to prioritizing safety, building the Claude chatbot along the way, has been one of the loudest voices raising warnings about how the same technology could bring catastrophe.
uncertain
technology → dedicate → catastrophe
It’s one of the contradictions that mark the brief, colorful history of Anthropic, which emerged Thursday from its first day of trading on Wall Street with a market value of $1.75 trillion.
asserted
which → ’ → trillion
The company, which still pledges to prioritize humanity’s benefit, now faces pressure from shareholders to wring profits from the technology its most senior leaders have described as posing existential threats.
asserted
leaders → pledge → threats
Anthropic CEO Dario Amodei described the push and pull between safety and AI advancement as one of the “central tensions at the heart of the company” on Thursday.
asserted
Amodei → describe → Thursday
In an interview with CNBC, he described the rapid pace of AI advancements — due in large part to advanced models helping to build the next, improved version of themselves — as akin to building something that lets a driver go 200 miles per hour when they had previously been driving at 60.
asserted
they → describe → 60
His company, he said, is working to develop it safely.
asserted
he → say → it
“What we’ve done with pacing is essentially said, ‘Look, you know, we could drive at 200 miles an hour.
uncertain
we → do → miles
Maybe let’s only drive at 90 miles an hour, spend time figuring out how to do that safely,’” he said.
asserted
he → let → that
“And the rest of that surplus, we spend building the model in the right way.”
asserted
we → spend → way
Warning of potential calamity, the company forges ahead
asserted
company → warn → calamity
The company’s seven founders — Amodei, 43, his sister and Anthropic’s president Daniela Amodei, 39, and five other former OpenAI employees — left the rival company in pursuit of building one that more clearly prioritized safety in artificial intelligence development.
asserted
that → leave → development
Anthropic publicly has been committed to safety and transparency around their AI models, but insiders have sounded alarm bells in recent months.
asserted
insiders → commit → months
An Anthropic researcher resigned from the company in September, criticizing the safety practices of both Anthropic and OpenAI.
asserted
researcher → resign → Anthropic
Jacob Coxon, whose message spurred industry-wide conversations about the speed of AI’s growth, wrote that both Anthropic and OpenAI are “gambling with our lives.”
asserted
Anthropic → spur → lives
He warned that some working on AI development believe it could threaten human life by the end of the decade.
uncertain
it → warn → decade
Dario Amodei responded with an essay calling for an industry-wide slowdown on the technology’s development.
asserted
Amodei → respond → development
He enumerated detailed plans for slowing development and ushering in coordination across different companies and countries, and championed the idea of having independent evaluators embedded within Anthropic and other labs.
asserted
evaluators → enumerate → Anthropic
But the results from at least one of those independent evaluations underscored Coxon’s concerns.
asserted
results → underscore → concerns
A recent report from the nonprofit Model Evaluation and Threat Research found Anthropic is developing its systems around twice as quickly as it was a year ago, largely because of the help of AI itself.
asserted
it → find → AI
Amodei said Thursday in comments at the Nasdaq opening bell that the company’s “most difficult safety work lies ahead of us.”
“And to state it unequivocally, we will pace the training and deployment of AI systems, taking as long as is necessary to have confidence that they are safe and will remain under human control,” he said.
asserted
he → say → control
A company pledging to benefit humanity has enriched its founders
Anthropic is a public benefit corporation, meaning it has a legal obligation to balance shareholder interests with the public good.
asserted
it → pledge → good
But these types of companies are rarely public, and of the few that are, they are significantly smaller than Anthropic, so the company is entering uncharted territory.
asserted
company → enter → territory
The company has said it is working towards a future where “AI’s benefits are shared by everyone,” but the initial public offering has created a concentration of extreme wealth for its founders, employees and other investors.
asserted
offering → say → founders
Before the IPO, the two Amodei siblings each had a net worth of just over $14 billion, according to Forbes.
uncertain
siblings → have → Forbes
The Amodeis and the other co-founders have said they are committed to pledging 80% of their equity to philanthropy, although that commitment is voluntary.
asserted
commitment → say → philanthropy
The company has an equity donation match program that allows employees to pledge equity, which Anthropic matches.
asserted
Anthropic → have → which
Anthropic has also pledged at least 5% of its stock to a “public benefit” fund.
asserted
Anthropic → pledge → fund
Speaking to CNBC Thursday, Daniela Amodei said the concept of a public benefit corporation “spoke” to the co-founders when they were initially creating the company.
asserted
they → speak → company
“We’re really trying to do something that we do think can have this incredible economic value and impact, but we want it to be baked into essentially our structure that we also have to balance that against building things that are good for people,” she said.
asserted
she → try → people
Anthropic offers itself as a solution for AI doomsday fears
In a filing with the Securities and Exchange Commission last month, Anthropic devoted an unusually lengthy 93 pages to the risk factors they must spell out for investors.
asserted
they → offer → investors
Among the concerns were that advanced AI systems could pose “catastrophic or existential risks to humanity.”
uncertain
pose → pose → humanity
Some argue Anthropic has outlined the ways its technology and that of its competitors could have devastating consequences in the name of sheer transparency.
uncertain
technology → argue → transparency
But critics have questioned the emphasis the company has placed on these dystopian-like potentials while positioning itself as the logical solution given its safety-minded focus.
asserted
company → question → focus
A “Saturday Night Live” sketch in September made this contradiction a punchline.
asserted
contradiction → make → September
Cast member Jane Wickline played Dario Amodei, saying: “Sam Altman, myself, all the major AI players, we have talked about this extensively, and we are all on the same page here: We do not condone what we are doing.”
asserted
we → play → what
The faux interview on “SNL” concluded with a line that seems to have resonated with skeptics, showing up on a protest sign outside of Anthropic’s San Francisco headquarters on Thursday: “I urge you to urge me to stop.”
asserted
I → conclude → me
Associated Press business writer Wyatte Grantham-Philips contributed.
asserted
Philips → contribute → ?