AI deepfakes used in schools and cryptocurrency scams
AI-generated deepfakes have been used to create sexually explicit images of students and teachers and to target school administrators with racist content, while also appearing in cryptocurrency fraud schemes. Schools across America are developing emergency response plans as lawmakers work to criminalize such misuse.
What happened
In separate incidents, school principals were depicted in AI-generated deepfake recordings spewing racist and violent language against Black students. One deepfake was produced by students; the other was created by a disgruntled athletic director who was arrested.
Schools are developing emergency response plans to address deepfakes, including the possibility of sexually explicit images of students or teachers generated by AI appearing on social media.
Reported impact
The rise of AI-generated deepfakes targeting administrators and students has prompted schools across America to struggle with the technology's harmful applications.
Deepfakes have emerged as a tool in cryptocurrency scams, with reports of deepfakes being used to target individuals and commit fraud.
Index analysis
Deepfakes are defined as non-consensually AI-generated voices, images, or videos created to produce sexual imagery, commit fraud, or spread misinformation.
According to a nonprofit research organization founder, technology to combat AI tools that replicate a person's image and voice does not yet exist, with both prevention and incident response requiring significant work.
Governance controls
On September 29, California enacted legislation targeting deepfakes.
State legislators are working to criminalize malicious deepfake acts specific to schools or children.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.