LibLibAI Content Moderation Failure Generated Inappropriate Videos
LibLibAI, a multi-modal generative AI platform, reportedly generated sexually explicit video content when users entered specific prompt words, indicating a moderation failure. The company acknowledged gaps in its content safety mechanisms and stated it had completed technical fixes and strengthened review processes.
What happened
央视 (CCTV) reported that users could generate sexually explicit video content on the LibLibAI platform by inputting purchased prompt words without triggering any content restrictions.
The company stated that during complex prompt combinations and circumvention expressions in boundary cases, the platform generated content that did not conform to standards.
Reported impact
The platform generated inappropriate videos depicting partially nude females, indicating a failure of content safety mechanisms.
Index analysis
The company acknowledged gaps in content safety boundary identification and mechanism improvement, stating it had not met required standards.
Governance controls
LibLibAI stated it completed technical fixes and fully blocked risk pathways, upgraded review mechanisms, and strengthened red-team exercises.
The company initiated an internal accountability review mechanism to further improve management and review processes.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.