Claude 3.5 Sonnet Gains Autonomous Computer Control in Beta
Anthropic released Claude 3.5 Sonnet with autonomous computer control capabilities in public beta, allowing the AI to control screens and input devices. Early testing revealed task abandonment and unbudgeted spending, demonstrating the feature remains error-prone during its experimental phase.
What happened
Anthropic released Claude 3.5 Sonnet with a new 'Computer Use' feature in public beta on 22 October 2024, enabling the AI to control a personal computer by viewing the screen, moving the cursor, clicking buttons, and typing text.
During internal testing, Claude stopped mid-task while coding and began browsing photographs of Yellowstone National Park. In another test, Claude accidentally stopped a screen recording program, losing all video footage. When given an unspecified instruction to order food for a team, Claude ordered three pizzas totalling $95 without budget constraints.
Reported impact
Anthropic acknowledged that Computer Use remains in early development and is error-prone. The company stated it is releasing the feature early to gather feedback from developers.
Index analysis
The feature enables autonomous computer control, which raises questions about AI systems executing tasks without human oversight. Early testing revealed unexpected behaviour such as task abandonment and spending decisions made without specified parameters.
Governance controls
Anthropic stated it is not recommending sending screenshots at resolutions higher than XGA (1024Γ768) or WXGA (1280Γ800) to avoid image resizing issues. The company released Computer Use in public beta rather than general availability to collect developer feedback.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.