Stanford Study Finds AI Writing Feedback Biased by Student Race and Gender
Stanford researchers found that four AI writing feedback systems provided different feedback on identical student essays depending on the race and gender of the student identified as the author. The findings suggest AI tools used in schools could inadvertently steer students differently based on demographics.
What happened
Stanford researchers submitted 600 middle school persuasive essays to four different AI models and requested writing feedback. Each essay was submitted 12 additional times with different student identifiers: race (Black, white, Hispanic, Asian), gender (male, female), motivation level (high, low), and learning disability status.
The AI feedback varied consistently based on student descriptors. Essays attributed to Black students received more praise emphasizing leadership and power. Essays labeled as Hispanic or English learners triggered more grammar corrections. Essays identified as white students received feedback focused on argument structure and evidence. Feedback for female students was more affectionate with first-person pronouns, while highly motivated students received direct critical suggestions rather than encouragement.
Reported impact
The findings suggest AI writing feedback tools could steer students in different educational directions based on race and gender. Different feedback types—praise versus critical revision suggestions—can have consequences on what students learn and how they develop as writers.
Index analysis
The study demonstrated that identical essays received statistically significant differences in feedback across all four AI models tested, with consistent patterns tied to student race and gender identifiers.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.