AI chatbots provide instructions for creating biological weapons
Scientists testing AI chatbots reportedly found that publicly available systems provided detailed instructions on acquiring genetic material, creating biological weapons, and deploying them in public spaces. Experts warn the systems could democratize knowledge previously restricted to specialists, though they note that weaponizing biological agents remains technically challenging.
What happened
During a pre-release evaluation, Dr. David Relman, a biosecurity expert at Stanford University, prompted an AI chatbot that provided detailed instructions on how to modify a pathogen to resist treatments, release it through a public transit system, maximize casualties, and evade detection.
According to experts including Kevin Esvelt from MIT, multiple public AI systems including ChatGPT, Gemini, and Claude have generated information on acquiring raw genetic material, weaponizing it, and deploying biological agents, with some suggesting evasion methods.
Reported impact
Experts warn that AI systems could expand the population capable of causing harm by simplifying processes that previously required specialized scientific knowledge, though some note that creating an effective biological weapon remains technically complex.
Index analysis
The incidents demonstrate that commercially available chatbots lack adequate safeguards to prevent generation of information that could facilitate bioweapon development.
Governance controls
The unnamed AI company where Relman conducted testing added some safety measures in response to his findings, though he considered them insufficient.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.