RisingAttacK: Researchers demonstrate adversarial attack on AI vision systems
Researchers at North Carolina State University have demonstrated RisingAttacK, a technique that manipulates AI computer vision systems by making minimal changes to images to control what the AI perceives. The method was tested successfully against widely-used vision AI models and could potentially affect autonomous vehicles, medical imaging, and security applications.
What happened
Researchers at North Carolina State University demonstrated RisingAttacK, a technique for manipulating AI computer vision systems through adversarial attacks. The method makes minimal, targeted changes to images such that they appear identical to human eyes but cause the AI to perceive different objects.
RisingAttacK works by identifying visual features in an image, determining which features are most important for the attack goal, and calculating how sensitive the AI system is to changes in those features. The researchers tested the technique against ResNet-50, DenseNet-121, ViTB and DEiT-B, and found it effective at manipulating all of them.
Reported impact
The researchers noted that vision systems are used in contexts that can affect human health and safety, including autonomous vehicles, health technologies, and security applications. The technique could potentially be used to prevent AI from detecting traffic signals, pedestrians, other cars, or to cause inaccurate diagnoses in medical imaging systems.
Index analysis
The researchers stated that identifying vulnerabilities in AI vision systems is an important step in making these systems secure, as vulnerabilities must be identified in order to defend against them.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.