AI chatbots lack end-to-end encryption, exposing user conversations
Popular AI chatbots do not use end-to-end encryption and routinely collect user conversation data for training and monitoring purposes, with users unable to guarantee privacy of sensitive discussions. OpenAI's CEO confirmed conversations could be disclosed to courts in legal proceedings, and some users unknowingly make conversations public through share features.
What happened
Popular AI chatbots including ChatGPT, Copilot, and Gemini do not provide end-to-end encryption, which is standard in messaging apps like WhatsApp and Signal to ensure messages are encrypted on the sender's device and decrypted only on the recipient's device.
OpenAI CEO Sam Altman stated that if a user discusses sensitive personal information with ChatGPT and subsequently faces legal proceedings, the company could be compelled to provide those conversations to a court.
Reported impact
AI platforms routinely collect and store user conversation data to train and improve their models, potentially using data mining to analyze user behavior, identify trends, and monetize conversation content.
Some users are unaware that pressing a 'Share' button in applications like Meta's chatbot automatically makes their conversations public and visible in the 'Discover' feed, potentially appearing in Google search results.
Index analysis
Michael Veil, professor of digital law at University College London, explained that while some chatbots offer settings to limit use of conversations in future training, they retain all data for content moderation purposes to monitor harmful content.
Security experts warn that prompt injection attacks can deceive chatbots and extract sensitive user data, and integration of these systems into operating systems like Android and iOS makes them proximate to users' private files and messages.
Governance controls
Unlike conversations with lawyers or doctors, users do not have equivalent legal protections when discussing sensitive matters with AI chatbots, as confirmed by Altman's acknowledgment that conversations can be provided to law enforcement or courts.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.