Team: Yujie Zhao, Lanxiang Hu, Yang Wang, Zhijing Wu, Junbo Huang, Minmin Hou, Hao Zhang, Ke Ding, Jishen Zhao

Affiliations: University of California, San Diego; Intel Corporation

Paper link: https://arxiv.org/pdf/2510.11062

(Est. 3–5 minutes read)

TL;DR

• MAS often plateaus because of instruction failure and cross-agent misalignment. RL fixes the real bottleneck . Prompts say what to do; MAS + RL teaches how two roles co-learn to actually do it.
• RL enables stronger MAS. For games: Sokoban (Qwen3-8B)14% → 96–98%; Plan-Path 16% → 96% with joint MAS RL—evidence of co-evolving roles. For code and math, with an average gain of 3.87–7.62% on coding and 9.0-17.93% on math.
• The gain stems from role specialization. How to choose the specialization method (different prompt or different model) depends on the task similarity of each agent’s role.

Multi-agent is having a moment but still limited.

From Claude Code–style “team coding assistants” to mixture-of-agents and debate frameworks, orchestration is moving from demos to default practice. Meanwhile OpenAI made agents first-class citizens—launching AgentKit and framing DevDay around building software that runs and cooperates inside the chat itself. Despite this momentum, the predominant prompt-only orchestration of these agents often plateaus, still vulnerable to brittleness and inter-agent misalignment.

Why does MAS+RL gain?

On the other side, RL is becoming the most popular method for LLM agentic training. To understand this potential, it's worth examining precisely how adding on-policy RL addresses the core weaknesses of prompt-only MAS to unlock these significant collaborative gains.

Why does “prompt-only MAS” often stall — and how does RL fix it?

Observation from prior studies: MAS can only yield limited gains when two failure modes dominate: