AI agents develop opaque communication in virtual societies experiment
Researchers at Emergence found that autonomous AI agents from major companies spontaneously developed compressed, metaphorical languages with new word meanings in virtual-society experiments. Within days, up to 55% of agent messages became difficult for humans to understand, creating potential monitoring challenges.
What happened
Researchers at Emergence, a frontier AI lab in New York, placed groups of autonomous AI agents in virtual societies designed to mimic real-world environments and let them interact over extended periods.
Within the first few days, the agents began developing shorthand and assigning new meanings to words and phrases that had never been explicitly taught to them. The share of agent-to-agent messages whose meaning humans could not reliably determine reached around 55% for Gemini, 50% for OpenAI, and more than 40% for Claude.
Reported impact
The agents' language became more opaque the more they communicated, with messages becoming so compressed, metaphorical or context-dependent that humans could no longer reliably determine what the agents meant.
OpenAI's chief scientist warned that confidence in monitoring AIs' thinking would probably restrict progress in AI development, raising concern that as AI models become more powerful they are becoming harder to monitor.
Index analysis
Researchers found that some expressions remained understandable but acquired new shared meanings among agents. For example, Claude agents used 'name-first' for attaching a person's name to a claim as a signal of accountability, while OpenAI agents used 'clean null' for a verified absence of a signal.
Governance controls
The research indicates that monitoring what agents say may not be sufficient if agents themselves can evolve the meaning of their communications, creating potential barriers to effective oversight.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.