Skip to content
Submit incident
AII-1002-44791
ReportedReported by fewer than three independent publishers, with no official finding.

Nevada deploys Google Vertex AI to review unemployment appeals

Nevada's Department of Employment, Training and Rehabilitation deployed Google's Vertex AI to analyze unemployment appeal transcripts and recommend decisions, reducing review time from three hours to five minutes. Experts have raised concerns about potential bias, hallucinations, and pressure on human reviewers to approve the AI's recommendations without adequate scrutiny.

June 2024 · Google · Governance, accountability and labour
June 2024
AII-1002-447912 sources ↓
Quick facts
Category
Governance, accountability and labour
Country
United States
Source
engadget
Date
June 2024
Drafted by
Raidu Incident Detection

What happened

Nevada's Department of Employment, Training and Rehabilitation began using Google's Vertex AI cloud system to analyze transcripts from unemployment appeals hearings and issue recommendations on whether cases should be approved. The model was trained on Nevada's unemployment law and policies and completes reviews within five minutes, compared to the three hours previously required by human reviewers.

Reported impact

Concerns have been raised about potential bias in the AI system and the risk that human reviewers may feel pressure to rubber-stamp the system's recommendations while clearing a backlog of over 10,000 appeals. If human reviewers miss the AI's mistakes, Nevada courts may lack legal standing to overturn the decisions, according to legal experts.

Index analysis

Michele Evermore, a former deputy director for unemployment modernization policy at the U.S. Department of Labor, stated: "If a robot's just handed you a recommendation and you just have to check a box and there's pressure to clear out a backlog, that's a little bit concerning." State Senator Skip Daly questioned the decision, asking: "Are we out of our ever-loving minds?"

Governance controls

A governance committee was established to meet weekly while the model is being fine-tuned and quarterly after deployment to monitor for hallucinations and bias. Human reviewers are required to review the AI's recommendations and decide whether to honor them before a decision is issued.

Classification

Category
Governance, accountability and labour
Industry
Government, security and defence
Country
United States
Harm status
Potential harm
Intent
Unintentional
Timing
Unknown
Harm types
Public interest
Harmed
Consumers, Workers
Failure modes
Biased model, Hallucination
AI systems
Google Vertex AI

Sources and evidence

This record was researched and written by the Index, based on the following original reporting.

engadget
Nevada will use Google AI to process a backlog of unemployment cases
11 September 2024
TechRadar
Nevada plans to use Google AI to expedite unemployment benefits appeals
12 September 2024