2 weeks Ago By Kahekashan
Microsoft has rolled out a powerful new capability for its Copilot Studio: a feature called “computer use” that enables AI agents to directly interact with websites and desktop applications—just like a person would. This new functionality allows AI agents to automate tasks by clicking buttons, navigating menus, and entering data on the screen, even when there’s no API available. “Computer use enables agents to interact with websites and desktop apps by clicking buttons, selecting menus, and typing into fields on the screen,” explained Charles Lamanna, Microsoft’s corporate vice president for business & industry Copilot.
“This allows agents to handle tasks even when there is no API available to connect to the system directly. If a person can use the app, the agent can too.” With this upgrade, businesses using Copilot Studio can now build AI-powered agents that handle tasks like invoice processing, market research, and data entry more efficiently.
Microsoft says these agents can adapt to changes in the app interface, like button placements or screen layouts, without breaking down. Earlier this month, Microsoft also introduced a similar “Actions” feature for its consumer version of Copilot. This allows background automation of tasks such as booking restaurants, ordering products, or securing event tickets.
However, while “Actions” is currently limited to select partners, Copilot Studio’s version of “computer use” appears to be more flexible, supporting a wider range of websites and applications. This marks a big step forward in AI-driven task automation, expanding what digital assistants can do in real-world scenarios..
