French welfare agency's risk-scoring algorithm allegedly discriminates against marginalized groups
Amnesty International and 14 partner organizations filed a complaint with France's highest administrative court on October 15, 2024, alleging that the French Social Security Agency's National Family Allowance Fund (CNAF) uses a discriminatory algorithm to detect welfare fraud that disproportionately targets people with disabilities, single parents, and those living in poverty. The organizations claim the system violates European privacy rules and French anti-discrimination laws.
What happened
On 15 October 2024, Amnesty International and fourteen coalition partners led by La Quadrature du Net submitted a complaint to the Council of State, France's highest administrative court, demanding that CNAF stop using its risk-scoring algorithm.
Since 2010, CNAF has used the algorithm to identify people potentially committing benefits fraud by assigning risk scores between zero and one to all recipients of family and housing benefits. The algorithm processes personal data of 32 million people in France receiving CNAF benefits, as well as their family members.
Reported impact
The complainants allege the algorithm treats individuals experiencing marginalization—those with disabilities, lone single parents who are mostly women, and those living in poverty—with suspicion, violating the right to equality and non-discrimination and the right to privacy.
Risk criteria that increase scores include being on a low income, being unemployed, living in a disadvantaged neighbourhood, spending significant income on rent, and working while having a disability. Those flagged as high-risk are compiled into a list for further investigation by fraud investigators.
Index analysis
La Quadrature du Net obtained access to versions of the algorithm's source code in 2023, exposing the discriminatory nature of the system. The complainants argue the algorithm violates both European privacy rules and French anti-discrimination laws.
The legal challenge is described as the first time a public algorithm has been the subject of a legal challenge in France.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.