AI-generated malware targets French-speaking users
HP security researchers reported discovering a malware campaign targeting French-speaking users that was created using generative AI. The researchers identified AI-generated code through distinctive patterns including extensive comments and native language variable names, demonstrating that threat actors have bypassed AI safety measures to create malware at scale.
What happened
HP threat intelligence researchers identified a malicious campaign targeting French-speaking users where malware was developed using generative AI. The campaign, first detected in June 2024, used HTML smuggling to deliver a password-protected ZIP archive containing the malware.
Researchers determined AI was used to write the code based on the presence of comments throughout the malicious code, a rarity in human-written malware. The code structure, commented explanations of each line, and use of native language for function names and variables all pointed to AI generation.
Reported impact
Cybercriminals are using generative AI to write malware with increased speed and efficiency, bypassing restrictions placed in AI models. Threat actors have reportedly found ways around security measures that companies like OpenAI implemented to prevent malicious use.
Index analysis
The incident demonstrates that generative AI tools such as ChatGPT and Gemini, despite safety measures, can be exploited by threat actors to create malware more quickly and at scale. Low-level threat actors are increasingly leveraging AI to write and customize malware for attacks targeting various regions and platforms.
Governance controls
OpenAI updated ChatGPT's capabilities in 2023 to prevent threat actors from using it to write malicious emails or code, but researchers found that threat actors have bypassed these security restrictions.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.