AI
Google Gemini for macOS Gains Computer Control Capabilities
The AI assistant can now interact with local files, desktop applications, and web browsers to complete multi-step tasks autonomously.
Key takeaways
- Google Gemini for macOS now includes computer use capabilities that let the AI control apps, access files, and complete multi-step tasks autonomously.
- Users control which applications Gemini can access and can monitor its actions through a live preview window in the chat interface.
- The feature is rolling out now in English for US users, just ahead of Apple's new security restrictions for apps requesting full disk access.
- Apple's upcoming macOS restrictions specifically target potential risks from AI agents with computer use capabilities, appearing in the macOS 27.3 beta.
- Google suggests use cases including workflow automation, form filling, presentation creation, and e-commerce listing tasks.
Google has enabled computer use functionality in its Gemini app for macOS, allowing the AI assistant to directly interact with applications, files, and web browsers on behalf of users.
The new capability lets Gemini execute multi-step tasks that span across a user's local system, according to Google. The company positions the feature as useful for workflow automation, form completion, presentation creation, and e-commerce listing tasks.
How the feature works
Users maintain control over which applications Gemini can access through permission settings. The AI assistant operates through a dedicated background window that provides a live preview of its actions within the chat interface, allowing users to monitor what Gemini is doing in real time.
The feature is currently available in English exclusively for users in the United States, with Google beginning the rollout now.
Why it matters
The timing of this launch is notable. Apple is preparing to implement stricter controls around full disk access permissions on macOS, specifically citing security concerns related to AI agents with computer use capabilities. These new restrictions appeared in the macOS 27.3 beta released this week. Google's decision to roll out computer use functionality just ahead of these tightened security measures suggests the company wanted to establish its feature before the regulatory landscape shifts. For enterprise users considering AI assistants, this development highlights the evolving tension between AI automation capabilities and system security protocols.
Security and privacy considerations
The granular permission system represents Google's approach to addressing potential security risks. Rather than requiring blanket system access, users can selectively grant Gemini access to specific applications. This permission model may become increasingly important as Apple's forthcoming restrictions take effect.
The live preview window serves a dual purpose: it provides transparency into Gemini's actions while also giving users the ability to intervene if the AI assistant begins performing unintended operations.
Competitive landscape
Google's implementation of computer use in Gemini follows similar moves by other AI companies exploring autonomous agent capabilities. The feature represents a shift from conversational AI assistants toward systems that can take direct action on user devices.
The details were first reported by The Verge.