Google plans screen automation feature for Gemini to operate Android phones
Google is developing a screen automation feature for Gemini that will allow the AI to autonomously operate Android phones for tasks like ordering products or booking rides. The pre-deployment feature raises concerns about how Google will prevent the system from accessing or processing sensitive user data and unclear governance over AI actions on devices.
What happened
Google is developing a screen automation feature, codenamed 'bonobo', for Gemini that will allow the AI to autonomously operate Android phones by opening applications, scrolling screens and tapping buttons to perform tasks such as ordering products or scheduling rides in apps like Uber and Lyft.
The feature was first demonstrated at Google I/O through Project Astra, where Gemini showed capability to read on-screen content and react as a real user. Code for the feature has been detected in beta versions of the Google Android application.
Reported impact
Google has warned that Gemini may make errors and that users remain responsible for actions the AI performs. Screenshots taken during Gemini's operation of applications may be reviewed by trained reviewers if activity history storage is enabled.
Concerns have been raised about how Google will prevent Gemini from viewing or processing sensitive data such as login credentials or payment information, with unclear guidance on whether the system will automatically avoid such situations or require manual user intervention.
Index analysis
The feature represents a shift toward AI systems that not only advise but act autonomously on user devices, raising questions about control, privacy and responsibility that Google has not fully clarified.
Code hints suggest Gemini may track orders and purchases made on behalf of users, though these functions are not yet active.
Governance controls
Google states that users will be able to monitor what Gemini does, stop the system at any time and manually resume control. Users are advised not to input login data or payment information through Gemini conversations when activity history storage is enabled.
Classification
Sources and evidence
Sources not yet published
This record was researched and written by the Index from original reporting. Its sources are still being verified by an editor, and they are listed here once that check is complete. Until then the record carries no reviewer.