Alibaba open-sources Qwen-UI-Agent, GUI agent foundation model covering phone/desktop/web
On Aug 20 Alibaba Qwen released Qwen-UI-Agent, a GUI agent foundation model covering mobile, desktop, web and deep-search environments. Trained on 100+ real devices and 150+ apps, it beat GPT-5.6 Sol, Claude Opus 4.8, and Gemini 3.1 Pro on MobileWorld, WebArena and others, and carries built-in safety judgment plus 100+ step long-horizon tasks.
On August 20, Alibaba Qwen officially released Qwen-UI-Agent, a GUI agent foundation model aimed at real devices, covering mobile, desktop, web, and deep-search environments. The model aims to read screens, mimic human clicks and inputs across apps, and act as a general-purpose executor for digital devices — bypassing the need for software-vendors to expose APIs.
In benchmark performance: Qwen-UI-Agent scored 82.1 on MobileWorld, 92.2 on the real-device MobileWorld-Real benchmark, and 97.5 on AndroidDaily; 79.5 on OSWorld Verified; 73.6 on WebArena (rank 1); 81.5 on ScreenSpot-Pro element grounding. Multiple leaderboards were updated to SOTA and the model matched or surpassed GPT-5.6 Sol, Claude Opus 4.8, and Gemini 3.1 Pro. On OSWorld-v2 the model cut execution steps by 58 percent.
On data and methodology, the model was trained on Alibaba's large-scale real-device fleet of 100+ phones and 150+ apps, and the team released MobileWorld-Real, a 400-task real-device benchmark. The agent mixes GUI actions with command-line operations — nearly half of desktop actions invoke the CLI — which compresses execution traces. A complete safety module refuses illegal or high-risk requests outright, and pauses for user confirmation on sensitive actions like payments, deletions, and privacy authorizations.
Training supports online reinforcement learning on 100+ step trajectories with about 10,000 concurrent environments and an AutoResearch-style AI-driven data flywheel. Built on the Harness framework the agent proactively offers help, hands off tasks across phone and desktop, and pairs with DeepSearch to combine web retrieval and interface control. Technical report, project page, and code are all public.