1300 166 138 Tell us what you need
News / AI Agents

GitHub Copilot can now use the apps on your computer

By Monika Tantau • Published 2 months ago
Woman working at a desktop computer with a design app open, the kind of desktop software GitHub Copilot can now operate

Key takeaways

• GitHub opened computer use in Copilot on Mac and Windows as a public preview on 1 October
• Copilot can click, type, scroll and move between desktop apps on your behalf
• It is off by default and asks before it takes control of each app
• Admins can switch it off for a whole organisation through managed settings

Most Read

GitHub Copilot can now use the apps on your computer. On 1 October GitHub put a feature called computer use into public preview in the Copilot app and the Copilot command line tool on Mac and Windows, according to GitHub’s changelog. Once it is switched on, Copilot can read what is on screen, click buttons, type, scroll and drag, and work through a task that spans several programs.

This is GitHub Copilot, the AI assistant built for software developers, not the Microsoft 365 Copilot that sits inside Word and Outlook. But the new skill reaches well beyond code. In GitHub’s own demo, Copilot works its way through an expense report in a web browser.

What Copilot can now do on your desktop

Copilot reads each app through the operating system’s accessibility features and takes screenshots when it needs to see the layout, GitHub’s documentation explains. It then chooses its own actions as it works through your request, and every step shows up in the session so you can follow along.

The target is software that offers no other way in. GitHub names legacy programs and tools that only work through windows and buttons, with no API, no command line and no MCP connection. Its examples include summarising information in an old desktop application, updating a presentation and moving data from one program to another.

You stay in control, mostly

Computer use is off by default. Once it is on, Copilot asks for approval the first time it wants to control an app, and you can allow it for that session, save the approval or say no. On a Mac, it also walks you through the Accessibility and Screen Recording permissions it needs.

Choosing to always allow an app is where caution matters. That choice is stored on the computer and covers both the app and the command line tool, so Copilot will not ask again. GitHub advises against it for apps that hold sensitive information or can take high impact actions. If something goes wrong, the Escape key stops the current action.

Organisations keep the final say. Administrators can disable computer use through managed settings, and switching it on locally does not override that policy.

Laptop showing the Microsoft Copilot welcome screen; GitHub Copilot is a separate assistant built for developers

GitHub is upfront about the risks

The documentation is unusually blunt. Because Copilot interprets screens that change between versions and window states, it can click the wrong control, type in the wrong field or repeat an action. Unexpected content on screen can also lead to unintended actions that affect your data or connected accounts, including financial and enterprise systems.

GitHub’s own advice is to treat it as a last resort. If an API, an MCP server or a terminal command can do the job, that route gives more structured and predictable results. Computer use is for the screens nothing else can reach.

Why Australian businesses should care

Plenty of Australian businesses still run desktop software that was never built to connect to anything, from older accounting and practice management systems to tools built in house. Until now, automating them meant costly integration work or someone retyping data. Jobs like the expense report in GitHub’s demo are exactly the manual work that finance automation sets out to remove.

The catch is privacy. Copilot sees whatever is on screen, and GitHub notes that windows may show information about other people. For any business handling customer records, the Privacy Act 1988 still governs that information. The safer pattern is to approve one app at a time, keep sensitive systems off the always allow list and ask IT whether the feature should be disabled.

Desktop agents are arriving fast

The feature is still a preview, and GitHub says it may change. It follows Microsoft’s rebuild of its own assistant around agents on 25 September, including a Copilot that keeps working after you go home. The big vendors now want their assistants to act, not just answer.

For teams choosing a coding assistant, the gap is widening between tools that stay in the editor and tools that reach across the whole computer, a difference worth weighing in our Cursor vs GitHub Copilot comparison. The useful question for now is which repetitive desktop tasks you would trust it with, and which ones you would not.

Scroll to Top