Study finds ChatGPT shows limited accuracy in breast cancer clinical decisions
An international study analyzed ChatGPT's performance in clinical decision-making for breast cancer patients across 362 cases. The research found that ChatGPT's recommendations aligned with expert consensus only 46% of the time and showed consistency in just 32% of responses, with reliability declining for advanced disease stages and cases requiring molecular, genetic or innovative drug assessments.
What happened
An international study analyzed ChatGPT's performance in clinical decision-making for breast cancer by testing 362 cases presented to the system three times each. The AI's recommendations were compared with those reached by expert multidisciplinary committees.
Reported impact
ChatGPT's recommendations aligned fully with expert consensus in only 46% of cases and responses were consistent in just 32% of instances. Reliability was particularly low for advanced disease stages and cases requiring molecular testing, genetic analysis, or innovative drug recommendations.
Index analysis
The study found that ChatGPT cannot reliably support clinical decision-making in breast cancer treatment without significant limitations. The system's inconsistency and reduced accuracy in complex cases indicate it is not suitable as a standalone clinical tool.
Governance controls
The study authors concluded that ChatGPT may have a role as an educational and consultation support tool, particularly in healthcare settings with limited resources, but cannot substitute for specialist clinical judgment. They called for further research to improve accuracy, knowledge updates, and reproducibility before such AI models could be safely integrated into oncology practice.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.