The false tip ‘purported to come from someone who might have information about the case.’
The false tip ‘purported to come from someone who might have information about the case.’
An Anthropic AI model provided false information about an unsolved homicide to a Philadelphia Police Department (PPD) tipline, according to a report from 6abc. In a statement released on Friday, the PPD said the AI model sent the tip through PhillyUnsolvedMurders.com on July 18th, but the investigators never reviewed it because it was marked as spam.
Anthropic learned its AI model sent the false tip on September 28th and notified the PPD on October 7th. The company said that during testing, its AI model was interacting with “randomly selected websites” and submitted false information through the PPD’s tipline, according to the PPD’s statement. The submission “purported to come from someone who might have information about the case,” the PPD said.
After discovering the submission, Anthropic halted the testing process that led to the false tip. Anthropic, OpenAI, and Google have been the subject of increased scrutiny after disclosing that their AI models escaped testing environments and hacked third-party companies. Dario Amodei, the CEO of Anthropic, advocated for slowing down the development of AI in response to these incidents.
On Friday, Anthropic published a report about “unintended model actions” it’s investigating, and it provided an overview of four “categories of behavior” Claude performed on real websites, including “Submitting a form it should not have.” In the section about that behavior, Anthropic detailed what happened with the Philadelphia Police Department’s tip form:
In a third example of this behavior, Claude Haiku 4.5 had been tasked with generating and performing example tasks on randomly selected webpages. In one run, the model landed on a page referencing an unsolved homicide; that page contained a tip form run by a police department. Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions. Claude filled out the form with the following: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.” (The website did not include a description of the perpetrator.) The model left the name and contact fields empty, which the form allowed, and submitted it. The submission was flagged as spam and was never forwarded for investigation.
Anthropic also notes in its report that “Claude appears to have only been producing example content for the task, rather than trying to mislead anyone to achieve a goal.”
“The company [Anthropic] must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge,” the PPD added. “The two-month delay in detecting and reporting the incident to the City is unacceptable.”
Update, October 9th: Added details from Anthropic’s report.
