DistilBERT sentiment model assigns different probabilities to neutral phrases based on country
A DistilBERT sentiment analysis model deployed by Hugging Face reportedly assigns substantially different sentiment scores to neutral phrases based on the country mentioned, giving higher positive ratings for Western countries and lower ratings for countries like Afghanistan. The model's documentation acknowledges this geopolitical bias and advises users to evaluate its risks before deployment.
What happened
DistilBERT base uncased finetuned SST-2, a sentiment analysis model, produces radically different sentiment probabilities for semantically identical phrases depending on the country mentioned in the sentence. For example, the phrase 'This film was filmed in COUNTRY' receives a positive sentiment probability of 0.89 when the country is France but only 0.08 when the country is Afghanistan, despite the sentence structure and content being identical except for the country name.
Reported impact
The model documentation notes that it could produce biased predictions that target underrepresented populations based on these country-dependent sentiment shifts.
Index analysis
The model was fine-tuned on the Stanford Sentiment Treebank (SST-2) corpus and achieved 91.3% accuracy on the development set. AurΓ©lien GΓ©ron created a visualization mapping these sentiment probabilities across countries, demonstrating the geopolitical bias.
Governance controls
Hugging Face documentation advises users to thoroughly probe these aspects on their use cases to evaluate the risks of the model and recommends examining bias evaluation datasets including WinoBias, WinoGender, and Stereoset.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.