Sora video generator found perpetuating sexist, racist, ableist stereotypes
A WIRED investigation of hundreds of videos generated by OpenAI's Sora found the system perpetuates sexist, racist, and ableist stereotypes, depicting CEOs and professors exclusively as men, flight attendants as women, with limited racial diversity and limited representation of disabled and overweight people. OpenAI acknowledged the issue and cited ongoing efforts to reduce bias through training data adjustments and moderation techniques.
What happened
A WIRED investigation involving a review of hundreds of AI-generated videos found that Sora frequently produces biased outputs. Pilots, CEOs, and college professors are depicted exclusively as men, while flight attendants, receptionists, and childcare workers are women. Disabled people are consistently portrayed as wheelchair users, interracial relationships are difficult to generate, and fat people do not run.
Prompts for gay couples mostly returned conventionally attractive white men in their late 20s with the same hairstyles. The investigation tested outputs for generic actions such as 'a person walking' and job titles, revealing limited racial diversity and ignored prompts for broader representation.
Reported impact
Women, people of color, and individuals with disabilities face exclusionary narratives and stereotypes from Sora's outputs. At the moment, the most likely commercial use of AI video is in advertising and marketing; if AI videos default to biased portrayals, they may exacerbate the stereotyping or erasure of marginalized groups.
Society faces the risk of heightened unrepresentative and discriminatory information from Sora and equivalent products. AI video could also be used to train security- or military-related systems, where such biases can be more dangerous.
Index analysis
The biases stem from Sora's training data, which reflects societal prejudices embedded in the datasets. The issue largely stems from how these systems work by absorbing large amounts of training data—much of which reflects existing social biases—and seeking patterns within it.
Research on image generators has found that these systems do not just reflect human biases but amplify them. Other choices made by developers during the content moderation process can ingrain these biases further.
Governance controls
OpenAI acknowledged the issue and cited ongoing efforts to reduce bias through adjustments in training data and moderation techniques. OpenAI's system card acknowledges that biased representations are an ongoing issue with the model, though the researchers believe that 'overcorrections can be equally harmful.'
OpenAI has safety teams dedicated to researching and reducing bias and other risks in its models. The company researches how to change its training data and adjust user prompts to generate less biased videos.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.