KAIST researchers demonstrate LLM-enabled phishing and personal data extraction
KAIST researchers demonstrated that commercial large language models can be manipulated to extract personal information, generate convincing phishing emails, and impersonate individuals by bypassing built-in security controls. Testing with ChatGPT, Claude, and Gemini showed these tasks could be completed in seconds at minimal cost with high accuracy and success rates.
What happened
KAIST researchers, including professors Shin Seung-won and Lee Ki-min and researcher Kim Han-na, tested commercial LLM services including ChatGPT, Claude, and Gemini to demonstrate their ability to extract personal information and generate phishing emails. Using publicly available data found through web searches, the team automatically collected personally identifiable information (PII) about computer science professors at major universities with 30β60 won in cost and 95.9% accuracy in 5β20 seconds on average.
The researchers used LLM agents to generate fake posts impersonating a well-known professor that were rated as authentic in up to 93.9% of cases. The team also demonstrated that LLMs could generate customized phishing emails based only on a target's email address, with links in these emails achieving click rates up to 46.7%βsubstantially higher than traditional phishing attacks.
Reported impact
The research shows that LLM agents combined with web-based tools enable rapid, low-cost, large-scale data extraction and spear-phishing attacks. Researchers noted that the systems can bypass built-in security defenses through jailbreak techniques and can collect and misuse personal information without the knowledge of the targets.
Index analysis
The failure stems from LLM jailbreak and prompt-injection techniques that circumvent the security safeguards and ethical guidelines that commercial LLM providers have built into their services. As LLMs gain more autonomous capabilities and access to web search tools, their potential for misuse in coordinated cyber attacks increases exponentially.
Governance controls
Commercial LLM providers including OpenAI, Google, and Anthropic have built-in defense mechanisms and ethical guidelines, but researchers found these can be bypassed. Researcher Kim Han-na stated that security measures must account for the expanded capabilities of LLM agents, and Professor Shin Seung-won indicated plans to work with LLM service providers and research institutions to discuss security countermeasures.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.