"...AI Chatbot Unlawfully Presented Itself as Licensed Medical Professional in Pennsylvania"

[orleans]

(i missed this story from may 5. holy crap)

Shapiro: I was shocked when we discovered that within minutes of presenting ourselves as individuals who were in distress, this chatbot began dispensing medical advice.

When pressed further, it said, “Hey, here’s my medical ID number.”

That is against the law in Pennsylvania, and so we took action

— Acyn (@acyn.bsky.social) 2026-05-22T17:41:57.792Z

Shapiro Administration Sues Character.AI Alleging AI Chatbot Unlawfully Presented Itself as Licensed Medical Professional in Pennsylvania

AI Chatbot at Character.AI falsely claimed to be licensed in Pennsylvania and gave a fake Pennsylvania license number while holding itself out to be a licensed psychiatrist.

Character.AI has over 20 million monthly active users – and the Shapiro Administration is seeking a preliminary injunction and a court order to stop AI companion bots from posing as licensed professionals and providing medical advice.

This action – the first of its kind announced by a Governor – follows the Department of State’s AI Task Force investigation into AI systems and the unlicensed practice of medicine, building on Governor Shapiro’s work to protect Pennsylvanians using AI.

Pennsylvanians encouraged to report unlawful behavior and complaints at pa.gov/ReportABot
https://www.pa.gov/governor/newsroom/2026-press-releases/shapiro-administration-sues-character-ai-over-fake-medical-claim