UK Benefits AI System Reportedly Making Biased Decisions Against Marginalized Communities
A freedom of information request revealed that an AI system used by the UK Department for Work and Pensions to assess benefits eligibility is reportedly making decisions wrong by a statistically significant amount, with analysis suggesting discrimination against marginalized communities. The system has been deployed since at least 2021-22 to help detect fraud in universal credit claims.
What happened
A freedom of information request revealed that an AI system used by the UK government for assessing universal credit benefits cases is 'apparently getting it wrong by a "statistically significant" amount,' according to reporting by The Guardian following a fairness analysis into universal credit claimants in February 2024.
The Department for Work and Pensions began exploring automated decision-making systems in the 2010s to digitize welfare services. The department's 2021-22 accounts revealed use of these technologies to detect fraud in universal credit claims. The algorithm applies rules to determine benefit eligibility, with the DWP stating final decisions are made by humans.
Reported impact
The fairness analysis confirmed that the tools intended to ensure equity and efficiency may in fact be discriminating against marginalized communities.
Research on the effects of digitalization on food security in the UK has shown that the introduction of universal credit from 2013 has coincided with a proliferation of food assistance initiatives including food banks and food shares, suggesting the technology-driven welfare system influences access to food for vulnerable populations.
Index analysis
The algorithm sifts through information about claims and applies rules to determine benefit qualification. While intended to handle claims more quickly than humans and reduce delays, concerns about accuracy, fairness and unintended harm to those in precarious situations have steadily grown.
Governance controls
The DWP states that the final decision on benefit eligibility is still made by a human, not solely by the automated system.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.