Tech firms develop tools to detect AI-generated deepfakes and synthetic media
Scammers worldwide have used AI-generated deepfakes and voice clones to conduct fraud against families and businesses, with reported losses including a 200 million Hong Kong dollar transfer. Tech companies have developed detection tools to identify synthetic media, though effectiveness varies and deepfake generation tools are proliferating rapidly.
What happened
Scammers have used AI-generated voice clones and deepfake videos to conduct fraud. In California, a woman received a fake voice call from someone impersonating her daughter, claiming to be in a hospital accident. In February, Hong Kong police reported that an employee of a multinational company was persuaded to transfer 200 million Hong Kong dollars to fraudsters who conducted a video conference using life-sized AI avatars impersonating his colleagues.
Tech companies have developed detection tools to combat these threats. Honor released the Magic 7 smartphone with real-time detection of AI use in video calls. Surf Security launched a browser that alerts users when audio or video is AI-generated. Intel uses facial blood vessel analysis in its Fake Catcher tool, while Bindrop Security analyzes audio samples for human voice characteristics. Reality Defender and Attestiv also offer real-time detection systems.
Reported impact
A study by iBoom in February found that only 0.1% of Americans and British tested could correctly identify a fake image or video. According to Vijay Balasubramanian of Bindrop Security, there are now 490 AI voice generation tools available, compared to just one a decade ago.
Deepfake technology has become a global cybersecurity threat affecting all sectors. Nicos Fekiaridis of Attestiv stated that companies' reputations can be damaged by deepfakes, and fraud extends beyond traditional financial and insurance sectors.
Index analysis
Researchers at the State University of New York at Buffalo suggest that deepfakes will become manageable like spam email once detection tools are widely deployed, similar to how email filters control unsolicited mail.
However, some academic publications have questioned the effectiveness of detection systems, while other university studies have highlighted high detection rates. Detection effectiveness varies, and the challenge increases as deepfake technology advances.
Governance controls
Multiple companies now provide AI-generated content detection capabilities to organizations. Vijay Balasubramanian indicated that all companies will eventually need to deploy AI content detection systems to protect against fraud and reputational damage.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.