OpenMobile-2 lifts a 27B phone agent from 31.9 to 59.6 on real Android devices
A 9 October paper from Nanjing University, SenseTime, and others, written up by The Decoder on Sunday: more than 110 practice apps, nearly 12,000 demonstrations taught by Gemini 3.1 Pro, and a 27B model that reaches 79.9 on AndroidWorld. Reinforcement learning raised the real-device score and did not raise AndroidWorld. The weights and the gym are open. The scores are the authors’.
Feature3 minIn-depth
James Okonkwo3 min read
- Android
- Agents
- Qwen
- SenseTime
- Open weights

Photo: Yan et al., arXiv:2610.14157Source page
Researchers at Nanjing University, SenseTime, and several other universities have released OpenMobile-2, an open phone agent and the gym they trained it in. The paper, posted to arXiv on 9 October as 2610.14157, says the training playground spans more than 110 apps. Thirty-five of them are simulated commercial apps in a system they call MobileGym++, with cross-app workflows that can be reset and checked by a program. The app tables include Taobao, JD, DiDi, Meituan, Slack, and Google Drive. The Decoder’s Jonathan Kemper wrote the paper up on Sunday.
The data, as the abstract counts it, is nearly 12,000 mobile interaction trajectories for supervised fine-tuning, and more than 2,000 executable tasks with automatic rewards for reinforcement learning. Gemini-3.1-Pro-Preview writes the user instructions and, as the teacher, carries them out. Only rollouts that finish the task and pass a verifier are kept. The agent, the paper says, sees screenshots and not the app’s internal data, which is the constraint a person tapping a phone also has. They train two sizes, from Qwen3.5-9B and Qwen3.6-27B.
The 27B scores, and the stage that did not help
The headline numbers in the abstract are the ones after reinforcement learning. On AndroidWorld the 27B model reaches 79.9, up from 67.1 for the Qwen3.6-27B base, a gain the paper’s text puts at 12.8 points. On MobileWorld, a harder set of longer tasks, it reaches 50.4, up from 35.9. On real Android phones, SPA-Bench moves from 31.9 for that base model to 59.6. Table 2 splits the last step. Hybrid supervised fine-tuning already has the 27B model at 80.8 on AndroidWorld and 57.4 on SPA-Bench. Reinforcement learning, run from that checkpoint, takes SPA-Bench to 59.6 and AndroidWorld to 79.9. The extra stage raises the real-device score. It does not raise AndroidWorld. The paper’s own sentence is that reinforcement learning helps where the training environments and the test look alike, and that gains elsewhere stay limited.
A separate cut holds the amount of training data fixed at 2,000 emulator trajectories and changes only how many apps those trajectories come from, from 15 apps to 53. Pass@1 on applications the agent has not seen moves from 60.7 to 67.5. Pass@1 on the familiar apps stays roughly flat, 73.7 to 71.4. The paper’s claim is that variety of apps, not more examples from a few of them, is what moves the unseen-app number.
Shortcuts the model ignores until it is trained to use them
The experimental half is a hybrid. More than 50 simulated apps expose functions as tools, on the order of 300 tools, so the agent can call something like a search or an add-to-cart instead of tapping through menus, and can still fall back to the screen. The paper says the tools do little until the model is trained on them. Before that hybrid fine-tuning, tool calls appear in 20 to 102 of the 215 tasks on the authors’ own bench, depending on the model. After it, the gains from having the tools are 5.6 points for the 9B model and 10.3 points for the 27B, and the trajectories get shorter by 4.8 and 5.8 steps. On MobileWorld’s tool setting, whose tools were not in the training set, hybrid fine-tuning scores 45.0 for the 27B model against 35.0 for a GUI-only fine-tune. The authors call the tool interface experimental, and it exists in the simulations, not as a switch a phone owner can flip.
The comparisons with closed models are the authors’ reproductions, marked in the table as run outside their own harness. In that column Gemini-3.1-Pro-Preview scores 70.7 on AndroidWorld and 70.2 on SPA-Bench. OpenMobile-2’s 27B model is ahead of the Gemini figure they print on AndroidWorld, 79.9 against 70.7, and behind it on the real-device bench, 59.6 against 70.2. The weights, the gym, and the tasks are what they are releasing. The scores are theirs.


