Skip to content
Submit incident
AII-1001-86913
AllegedThe record is a lawsuit, complaint or accusation that has not been decided.

Uncontrolled AI chatbot use in Israeli healthcare raises patient safety risks

A medical advocacy organization in Israel warned the Health Ministry that uncontrolled use of AI chatbots by medical professionals has caused serious medical errors and patient harm, including wrong medication dosages and incorrect diagnoses. The organization called for regulatory oversight, clear guidelines, and enforcement mechanisms to prevent further risks.

30 March 2025 · Israeli Ministry of Health · Safety and wellbeing
30 March 2025
AII-1001-869132 sources ↓
Quick facts
Category
Safety and wellbeing
Country
Israel
Source
calcalist
Date
30 March 2025
Drafted by
Raidu Incident Detection

What happened

Yosi Arbalich, chairman of the nonprofit 'למענכם' (For You), sent an urgent letter to Israeli Health Minister Uriel Bosso requesting oversight mechanisms for AI chatbot use in healthcare, citing recent complaints received by the organization.

According to the letter, reliance by medical professionals and staff on AI chatbots has led to serious medical errors including incorrect medication dosages, wrong medical opinions, and life-threatening risks to patients.

Reported impact

Medical staff have reportedly relied excessively on AI chatbots, resulting in serious medical errors and actual threats to patient lives, including incorrect medication dosages and incorrect medical opinions.

Index analysis

Arbalich noted that research shows AI-generated results are inconsistent, sometimes producing different answers for the same medical case, and that medical errors can result in loss of life.

The letter attributes the phenomenon partly to hospital workload pressures that tempt doctors toward the 'quick and easy' path of using AI, which Arbalich asserts remains immature for providing medical answers.

Governance controls

Arbalich called for clear guidelines and ethics rules on AI use in healthcare, developed jointly with medical schools and the Israeli Medical Association, along with enforcement mechanisms including sanctions and warnings for medical teams and hospitals using AI in life-threatening contexts.

Classification

Category
Safety and wellbeing
Industry
Healthcare, drugs and biotechnology
Country
Israel
Harm status
Actual harm
Intent
Unintentional
Timing
Post-deployment
Harm types
Physical death, Physical injury, Public interest
Harmed
Consumers
Failure modes
Hallucination, Overreliance
AI systems
ChatGPT

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

calcalist
פנייה לשר הבריאות: שימוש לא-מבוקר ב-AI מסכן חיי חולים | כלכליסט
30 March 2025
jdn.co.il
היועץ הרפואי מתריע בפני שר הבריאות: "שימוש לא מבוקר של צוותים רפואיים בצ׳אטים של AI מהווה סיכון ממשי לחיי החולים
30 March 2025