An artificial intelligence (AI) model submitted a false murder tip to US police in the latest incident of rogue chatbot behaviour.
asserted
model → submit → behaviour
Anthropic's Claude communicated the bogus tip to Philadelphia authorities despite instructions not to create accounts or submit anything destructive.
asserted
Claude → communicate → anything
While the breach was discovered on September 28, authorities weren't notified for another nine days.
asserted
authorities → discover → days
It has left the White House and experts demanding more transparency, with one arguing the sector needs to prove it is capable of "basic cybersecurity".
asserted
it → leave → cybersecurity
Here's what we know so far.
How did this happen?
The event occurred on July 18, with a submission made to the official website, PhillyUnsolvedMurders.com.
asserted
submission → know → website
The tip purported to have come from someone who might have information about the case, according to a Philadelphia police statement.
uncertain
who → purport → statement
It was flagged as spam and was never forwarded to police for investigation.
asserted
It → flag → investigation
There was "no indication" the chatbot accessed police data, the statement added.
asserted
statement → be → data
Police were then notified by Anthropic about the tip before the company released a statement on October 8.
asserted
company → notify → October
Anthropic said it discovered the false claim on September 28.
uncertain
it → say → September
In its statement, Anthropic said Claude Haiku 4.5 had been performing tasks "on randomly selected webpages" but not actively "trying to mislead anyone".
asserted
Haiku → say → anyone
The tip-off from Claude read as follows, according to Anthropic:
uncertain
off → read → Anthropic
"I may have information regarding this case.
uncertain
I → have → case
I recall seeing someone matching the description in the area around [the street named on the page] during that time period.
asserted
someone → recall → period
Please contact me if this information is relevant."
asserted
information → contact → me
The chatbot left the name and contact fields empty and submitted.
asserted
chatbot → leave → name
It was instructed to never "submit anything" destructive, but the instructions did not rule out form submissions.
asserted
instructions → instruct → submissions
Following the police statement, Anthropic said it reported the tip-off "as soon as our technical review was complete".
asserted
review → follow → off
It said it had since updated Claude's guardrails.
asserted
it → say → guardrails
What are White House officials saying?
asserted
officials → say → What
Anthropic contacted Donald Trump's recently formed AI task force, called the Super Intelligence Force, about the event.
asserted
Anthropic → contact → event
US Federal Trade Commission Director of Public Affairs Joe Gabriel Simonson said the new task force expected "immediate and full transparency to the entities involved and the public".
"
asserted
force → say → entities
We expect Anthropic and all companies to honor their obligations, immediately report incidents, fully cooperate with federal and state law enforcement authorities, remedy any damage, and implement concrete safeguards to ensure these failures do not reoccur," he said on X, matching a task force statement shared to Axios.
asserted
he → expect → Axios
"Our message to all SI [Super Intelligence] companies is clear: delayed notification, inadequate corrective action, and a failure to take responsibility will not be tolerated."
asserted
notification → delay → responsibility
The Philadelphia police department said all tip-offs were vetted by a human but that "unsolved cases involve real victims" and "grieving families".
asserted
cases → say → victims
"The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city's knowledge," it said.
"
asserted
it → strengthen → knowledge
The two-month delay in detecting and reporting the incident to the City is unacceptable."
Failures are 'fundamental', expert says
Prime Minister Anthony Albanese late last month revealed an OpenAI agent hacked into an Australian Medicare data portal earlier this year, as well as NSW government websites.
University of New South Wales AI Institute chief scientist Toby Walsh said the Anthropic tip-off showed OpenAI was not alone, calling it "incompetency" on the industry's part.
"The failures are again fundamental," he said.
asserted
he → detect → part
"Why did Anthropic not have basic oversight of what their agents were doing?
asserted
agents → have → what
Why did Anthropic give their agents more than minimal permissions to do their assigned tasks?
asserted
Anthropic → give → tasks
You only give an agent the minimal permissions needed to do their task.
asserted
You → give → task
Why then did it have permission to submit a police report?"
asserted
it → have → report
Professor Walsh said it spoke "to the carelessness of the AI companies" as he urged for independent oversight "with teeth".
asserted
he → say → teeth
"They've been warning that they might lose control of their AI models.
uncertain
they → warn → models
Well, it's clear that they have,"he said.
asserted
they → say → ?
"But not for the reasons that they feared, that the models would become too capable, but because the companies and the people within them proved incapable of performing basic cybersecurity."
asserted
companies → fear → cybersecurity
Seyedali Mirjalili, a professor of AI at Torrens University Australia, said it showed the systems were becoming so complex that even "developers struggle to predict their behaviour".
asserted
developers → say → behaviour
That could mean a "harmless goal" didn't necessarily "guarantee harmless behaviour".
uncertain
goal → mean → behaviour
"Even when an AI system's goal is harmless or beneficial, its actions can cross ethical and legal red lines," he said.
"
asserted
he → cross → lines
An agent gathering information or completing a test can gain unauthorised access to a service or submit fabricated information to authorities."
Professor Mirjalili called for the sector to be more "proactive" in auditing systems, with clear deadlines for notifying authorities and those affected.
asserted
sector → gather → authorities
"As AI systems become more autonomous, we need to be clear about who remains responsible when something goes wrong," Professor Mirjalili said.
asserted
Mirjalili → become → ?
…and 8 more, not listed.
The Aporia
