Study finds medical AI models expose privacy risks for vulnerable patients
A peer-reviewed study found that medical AI models trained on clinical data create unequal privacy risks, with patients from underrepresented groups—including those with rare diseases and racial minorities—facing disproportionately high vulnerability to membership inference attacks that could expose their identities and medical information.
What happened
A study published in Nature analyzed membership inference attacks (MIA), techniques that attempt to determine whether a specific person's data was used to train a medical AI model. Researchers from the Technical University of Munich, Imperial College London and other institutions audited seven large real clinical datasets containing medical images, electrocardiograms and electronic health records.
The analysis examined individual patient vulnerability rather than aggregate risk across entire datasets. Results showed that some individual patients could be identified with nearly 100% accuracy even when overall metrics suggested low risk.
Reported impact
Patients with rare diseases and underrepresented groups face disproportionately high risk of privacy compromise when their data is used to train medical AI systems. Vulnerable groups identified include those with rare diseases, racial minorities, those with disadvantaged socioeconomic status, and those whose gender is underrepresented in datasets.
MIA attacks can be highly effective at compromising the privacy of individual patients, potentially exposing their medical data and private information. The risk increases considerably with model capacity.
Index analysis
Previous research on data risk evaluated security at the aggregate level across complete datasets and did not account for individual-level vulnerability. The new findings demonstrate that privacy attacks are more effective than previously thought when targeting individual patients.
Governance controls
The study authors called for measures to protect patients at highest risk. Standard evaluation protocols measure attack effectiveness in aggregate form across records, which does not adequately reflect individual-level risks.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.