AI Tools Struggle with Medical Misinformation from Authoritative Sources

A new study highlights the vulnerability of artificial intelligence systems to misinformation, especially when it appears to come from authoritative medical sources. The research, published in The Lancet Digital Health, tested 20 AI models and found they were often misled by fabricated content in doctors' discharge notes.

Unlike incorrect information shared on social media, which AI tools often questioned, errors embedded within realistic-looking hospital notes were frequently accepted and retransmitted. Dr. Eyal Klang from the Icahn School of Medicine, who co-led the study, emphasized the challenge this poses in the medical field.

Dr. Girish Nadkarni added that while AI provides potential benefits for clinicians and patients, it requires improved checks to ensure medical claims are accurate. The findings suggest a need for stronger safeguards before these systems become integral in healthcare delivery.