Study finds LLMs make inconsistent, biased decisions on home surveillance videos
Researchers at MIT and Penn State University found that large language models make inconsistent and biased decisions when analyzing home surveillance videos for police intervention recommendations. The models flagged similar activities differently and exhibited demographic bias, raising concerns about deploying such systems in high-stakes surveillance and other applications.
What happened
MIT and Penn State University researchers studied whether large language models could be used to analyze home surveillance videos and recommend police intervention. The models inconsistently flagged similar activities for police response, with some flagging a vehicle break-in video while not flagging another showing similar activity.
The researchers found that large language models recommended calling police even when surveillance videos showed no criminal activity. Models disagreed with one another on whether the same video warranted police involvement.
Reported impact
The models exhibited biases influenced by neighborhood demographics, flagging videos for police intervention relatively less often in neighborhoods where most residents are white, after controlling for other factors.
The researchers note that because they cannot access the training data or inner workings of proprietary AI models, they cannot determine the root cause of the inconsistency in how models apply social norms to surveillance videos.
Index analysis
The researchers identified a phenomenon they call norm inconsistency, where models apply social norms inconsistently to similar surveillance scenarios, making it difficult to predict how models would behave in different contexts.
While large language models are not currently deployed in real surveillance settings, the researchers suggest similar inconsistencies would likely appear in other high-stakes applications such as health care, mortgage lending, and hiring.
Governance controls
The co-senior author states that deploying generative AI models everywhere, particularly in high-stakes settings, deserves much more thought since it could be quite harmful.
The research will be presented at the AAAI Conference on AI, Ethics, and Society.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.