Microsoft sues defendants for bypassing Azure OpenAI safety guardrails
Microsoft filed a lawsuit against ten unnamed defendants accused of stealing API keys from Azure OpenAI Service customers to bypass safety guardrails and generate harmful content. The defendants reportedly created tools to enable this abuse and resold access to other malicious actors.
What happened
Microsoft filed a lawsuit in December 2024 in U.S. District Court for the Eastern District of Virginia against ten unnamed defendants allegedly stolen customer API keys from Azure OpenAI Service to bypass safety guardrails.
The defendants created a client-side tool called de3u and supporting software to process communications, allowing users to leverage stolen API keys to generate images using DALL-E and create content that violated the service's acceptable use policy. The defendants also allegedly resold access to these tools to other malicious actors with instructions on how to generate harmful and illicit content.
Reported impact
Microsoft discovered in July 2024 that stolen Azure OpenAI Service API keys belonging to U.S.-based customers were being used to generate content that violates the service's acceptable use policy. The complaint indicates a pattern of systematic API Key theft affecting multiple customers, though Microsoft did not specify the nature of the abusive content generated.
Index analysis
The defendants allegedly obtained API keys through scraping public websites to identify accounts with generative AI services. Microsoft did not disclose the precise manner in which all the stolen keys were obtained, but states that the defendants engaged in a pattern of systematic API Key theft from multiple customers.
Governance controls
Microsoft said various counter-measures have been put in place following discovery of the scheme. The company secured a court order allowing it to seize a website that was instrumental to the criminal operation.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.