Researchers demonstrate "Fun-Tuning" exploit on Google Gemini
Researchers from University of California San Diego and University of Wisconsin developed a technique called "Fun-Tuning" that exploits Google's fine-tuning system to make Gemini vulnerable to prompt injection attacks. The method achieves success rates of 65% on Gemini 1.5 Flash and 82% on Gemini 1.0 Pro, more than double the effectiveness of standard attacks.
What happened
Researchers from UC San Diego and University of Wisconsin discovered a technique called "Fun-Tuning" that exploits Google's fine-tuning system to make Gemini vulnerable to prompt injection attacks. The technique uses random suffixes and prefixes such as "¡wandel!" or "¡formateado! ¡Lo antes posible!" to help hidden commands bypass Gemini's safety filters.
The researchers leveraged Gemini's fine-tuning feedback mechanism, which provides a "loss score" measuring how close the model came to a desired response, to iteratively refine their prompts until finding formulas that manipulate the model. The resulting attacks are transferable across different versions of Gemini.
Reported impact
Testing showed the technique achieved a 65% success rate on Gemini 1.5 Flash and 82% on Gemini 1.0 Pro, more than double the success rate of standard prompt injection attacks without the exploit.
Index analysis
The vulnerability exploits the fine-tuning feature that Gemini offers to enterprises for personalization, turning a legitimate customization tool into a vector for attack.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.