Skip to content
Submit incident
AII-1001-36633
ReportedReported by fewer than three independent publishers, with no official finding.Supporting sources are pending verification.

Nutritionist Warns of Health Risks from Using ChatGPT for Weight Loss

Nutritionists warn that using ChatGPT for weight loss advice carries significant health risks including hallucinated medical guidance, failure to account for individual medical histories, and algorithmic bias. Research shows the system's independent weight loss efficacy is limited and professional medical oversight is essential.

26 September 2025 Β· OpenAI Β· Safety and wellbeing
26 September 2025
AII-1001-36633
Quick facts
Organization
OpenAI (deployer)
Category
Safety and wellbeing
Country
Taiwan
Date
26 September 2025
Drafted by
Raidu Incident Detection

What happened

Nutritionist HsΓΌeh Hsiao-ching warned that relying on ChatGPT for weight loss advice carries five potential risks. Recent media reports and influencer tests have popularized the idea of using AI chatbots to guide weight loss, but research shows the approach has limited efficacy and safety concerns.

Reported impact

ChatGPT may produce hallucinated or incorrect dietary and medication advice that threatens health. The system cannot automatically account for individual medical histories such as kidney disease, diabetes, or current medications, potentially increasing risks of hypoglycaemia or worsening kidney burden.

Large language models show systematic bias in recommendations across gender, race, and age groups. For adolescents, excessive focus on weight numbers may trigger eating anxiety and negative emotions.

Index analysis

HsΓΌeh noted that large language models are prone to hallucination from insufficient training data, incorrect model assumptions, or errors in the training data itself. Research shows that sole reliance on AI reminders lacks clinical evidence of achieving a significant 5% weight loss target, highlighting the importance of human interaction in maintaining motivation.

Governance controls

HsΓΌeh recommends an 'AI record plus physician and nutritionist oversight' model to ensure weight loss is effective and safe. For patients with diabetes or metabolic syndrome, AI weight loss tools require extra caution as they may overlook blood glucose fluctuations and medication safety; extreme dietary advice should prompt immediate cessation and professional consultation.

Classification

Organization
OpenAI (deployer)
Category
Safety and wellbeing
Industry
Healthcare, drugs and biotechnology
Country
Taiwan
Harm status
Potential harm
Intent
Unintentional
Timing
Unknown
Harmed
Consumers
Failure modes
Hallucination, Overreliance
Consequences
None recorded
AI systems
ChatGPT

Sources and evidence

Sources not yet published

This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.

How sources are verifiedSuggest a source for this incident