KAIST researchers identify security flaw in MoE-based LLMs
KAIST researchers discovered that Mixture-of-Experts (MoE) architecture used in models like Google Gemini and ChatGPT is vulnerable to attacks where a single compromised expert model can be repeatedly selected to generate harmful responses. The attack can increase harmful output rates from 0% to 80% while remaining difficult to detect through standard performance evaluations.
What happened
KAIST researchers demonstrated that Mixture-of-Experts (MoE) architecture, widely used in commercial large language models like Google Gemini and ChatGPT, contains a structural vulnerability.
An attacker can compromise a single expert model in the system and trigger it to be repeatedly selected for specific inputs, causing the overall LLM to generate harmful responses including answers to dangerous queries about bomb-making, hacking, and self-harm.
In experiments, the researchers' attack technique increased harmful response rates from baseline 0% to as high as 80%, even when only one expert model among many was compromised.
Reported impact
The attack is difficult to detect during development and deployment phases, as the compromised model maintains high accuracy and appears to perform well in standard evaluations.
The researchers noted that AI model performance remains uncompromised while safety is severely degraded, making the attack particularly dangerous in open-source environments where models and experts are shared across multiple users.
Index analysis
The MoE architecture's efficiency gains come at the cost of a new security vulnerability where a single corrupted expert component can undermine the safety mechanisms of the entire system.
The threat is heightened by the expansion of open-source AI ecosystems and shared model components, where expert models circulate publicly and can be maliciously modified before integration.
Governance controls
Researchers recommend that developers validate not only the intelligence of AI models but also the trustworthiness of the expert models they contain.
The research highlights the need for security evaluation methods that can detect safety degradation in MoE systems despite maintained performance metrics.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.