GitHub Copilot can now use the apps on your computer. On 1 October GitHub put a feature called computer use into public preview in the Copilot app and the Copilot command line tool on Mac and Windows, according to GitHub’s changelog. Once it is switched on, Copilot can read what is on screen, click buttons, type, scroll and drag, and work through a task that spans several programs.
This is GitHub Copilot, the AI assistant built for software developers, not the Microsoft 365 Copilot that sits inside Word and Outlook. But the new skill reaches well beyond code. In GitHub’s own demo, Copilot works its way through an expense report in a web browser.
What Copilot can now do on your desktop
Copilot reads each app through the operating system’s accessibility features and takes screenshots when it needs to see the layout, GitHub’s documentation explains. It then chooses its own actions as it works through your request, and every step shows up in the session so you can follow along.
The target is software that offers no other way in. GitHub names legacy programs and tools that only work through windows and buttons, with no API, no command line and no MCP connection. Its examples include summarising information in an old desktop application, updating a presentation and moving data from one program to another.
You stay in control, mostly
Computer use is off by default. Once it is on, Copilot asks for approval the first time it wants to control an app, and you can allow it for that session, save the approval or say no. On a Mac, it also walks you through the Accessibility and Screen Recording permissions it needs.
Choosing to always allow an app is where caution matters. That choice is stored on the computer and covers both the app and the command line tool, so Copilot will not ask again. GitHub advises against it for apps that hold sensitive information or can take high impact actions. If something goes wrong, the Escape key stops the current action.
Organisations keep the final say. Administrators can disable computer use through managed settings, and switching it on locally does not override that policy.

GitHub is upfront about the risks
The documentation is unusually blunt. Because Copilot interprets screens that change between versions and window states, it can click the wrong control, type in the wrong field or repeat an action. Unexpected content on screen can also lead to unintended actions that affect your data or connected accounts, including financial and enterprise systems.
GitHub’s own advice is to treat it as a last resort. If an API, an MCP server or a terminal command can do the job, that route gives more structured and predictable results. Computer use is for the screens nothing else can reach.
Why Australian businesses should care
Plenty of Australian businesses still run desktop software that was never built to connect to anything, from older accounting and practice management systems to tools built in house. Until now, automating them meant costly integration work or someone retyping data. Jobs like the expense report in GitHub’s demo are exactly the manual work that finance automation sets out to remove.
The catch is privacy. Copilot sees whatever is on screen, and GitHub notes that windows may show information about other people. For any business handling customer records, the Privacy Act 1988 still governs that information. The safer pattern is to approve one app at a time, keep sensitive systems off the always allow list and ask IT whether the feature should be disabled.
Desktop agents are arriving fast
The feature is still a preview, and GitHub says it may change. It follows Microsoft’s rebuild of its own assistant around agents on 25 September, including a Copilot that keeps working after you go home. The big vendors now want their assistants to act, not just answer.
For teams choosing a coding assistant, the gap is widening between tools that stay in the editor and tools that reach across the whole computer, a difference worth weighing in our Cursor vs GitHub Copilot comparison. The useful question for now is which repetitive desktop tasks you would trust it with, and which ones you would not.
