Kimsuky used AI-generated military ID deepfakes in phishing campaign
Cybersecurity researchers identified that the North Korean hacking group Kimsuky used ChatGPT to generate deepfake military ID images in a phishing campaign targeting South Korean researchers, human rights activists and journalists. The attackers used prompt injection to bypass ChatGPT's safeguards against creating military ID copies and embedded malicious code alongside the fake IDs to steal data from victims.
What happened
The North Korean threat actor group Kimsuky used ChatGPT to generate AI-created images of South Korean military ID cards in a phishing campaign. The fake ID images were attached to phishing emails that impersonated a South Korean defense-related institution, claiming to handle ID issuance tasks for military-affiliated officials.
The attackers overcame ChatGPT's refusal to generate military ID copies by using prompt injection techniques, framing the request as creating a mock-up or sample design for legitimate purposes. The email included a malicious batch file ('LhUdPC3G.bat') that, when downloaded, initiated malicious activity on victims' devices.
Reported impact
The AI-generated ID cards were identified as deepfakes with 98% probability. The campaign targeted researchers in North Korean studies, North Korean human rights activists, and journalists, and was designed to extract data from victims' devices and enable remote control.
Index analysis
The use of AI-generated images marked an evolution of Kimsuky's ClickFix phishing attacks previously observed in June. Genians researchers noted this as a real case demonstrating the group's application of deepfake technology.
Governance controls
ChatGPT is designed to refuse requests to produce copies of military government IDs. However, the report identified prompt injection as a technique to overcome this refusal by reframing the request as a mock-up or sample design rather than reproducing an actual military ID.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.