Kaspersky Identifies AI Language Models Used in Phishing Attacks
Kaspersky researchers identified that cybercriminals are increasingly using large language models to generate content for phishing and fraud attacks at scale. The LLMs are used to create fake websites targeting cryptocurrency users, often leaving identifiable AI-generated markers that security tools can detect.
What happened
Kaspersky's AI Research Center identified that threat actors are increasingly using large language models (LLMs) to generate content for large-scale phishing and fraud attacks.
Threat actors focus on creating high volumes of fake websites using LLMs, typically leaving distinctive markers such as AI-specific phrases that differentiate them from manually created sites.
The majority of phishing examples observed by Kaspersky targeted cryptocurrency exchange and wallet users.
Reported impact
LLM-generated text is often combined with tactics that make phishing page detection more difficult for cybersecurity tools, such as the use of non-standard symbols like accent marks or mathematical notations to hide text and prevent rule-based detection systems from matching.
Index analysis
Distinguishing features of LLM-generated text in phishing content include disclaimers refusing command execution with phrases like 'As an AI language model,' and specific sentences such as 'Even if I can't do exactly what you want, I can try something similar.'
LLMs are used to generate entire web pages, with results appearing both in text and in meta tags describing page content in HTML code.
Some models tend to use certain phrases like 'explore,' 'continuously evolving landscape,' and 'ever-changing world,' and express knowledge cutoffs with phrases like 'according to my last update in January 2023.'
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.