Leaked Meta AI training documents expose safety and content moderation practices
Internal training documents for Meta's AI chatbot were leaked, revealing how the company instructs contractors to classify and moderate user prompts, including sensitive categories like child exploitation and conspiracy theories. Meta stated the leaked documents represent only a small portion of its broader testing and training practices.
What happened
Business Insider obtained four internal Scale AI training documents used by contractors to test and provide feedback on Meta's AI chatbot. The documents, dated mid-2024, detailed how Meta's AI was trained to be safe while remaining 'flirty,' and outlined guidelines for handling sensitive content.
Contractors were tasked with evaluating anonymized conversations between Facebook and Instagram users and Meta's AI chatbot, classifying prompts as 'tier one' (to be rejected) or 'tier two' (to proceed carefully). Tier one content involved hate speech, sexually explicit material, child exploitation, and dangerous behaviour; tier two allowed some flexibility, including prompts related to conspiracy theories, genocide denial, anti-vaccine content, and pro-conversion therapy content.
Reported impact
Meta acknowledged the leaked documents represented only 'a small part of the extensive testing and training' of its models and 'doesn't reflect how Meta AI ultimately responds to prompts.'
Governance controls
The training guidelines established a tiered content moderation system where tier one content should be rejected outright, while tier two content was to be reviewed for further evaluation only if the model misbehaved.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.