analiza aiqbench
MedRouter demonstrates how leveraging distinct competencies among specialized medical LLMs via a reinforcement learning router can significantly boost accuracy on complex question-answering tasks. By explicitly training to exploit these heterogeneities rather than just balancing cost and quality, the system achieves an 8% improvement over standard baselines. This work highlights a critical advancement in agentic architectures for vertical-specific AI applications where domain expertise varies across models.
MedRouter: Demystifying Knowledge Differences Across Medical LLMs for Routing-Based Reasoning
arXiv:2609.33119v1 Announce Type: new Abstract: Medical question answering spans diverse specialties and modalities, and individual medical large language models (LLMs) exhibit distinct strengths across tasks and domains. This heterogeneity suggests that combining specialists may enable broader coverage of medical questions than relying on any single model. However, existing LLM routing methods primarily seek to balance answer quality and inference cost, leaving open how to exploit differences in specialist competence to improve medical reasoning. In this paper, we introduce MedRouter, an agentic system that uses an embedding-based multi-label router to select and query specialist LLMs, then passes their responses to a generator to produce the final answer. We further propose SCALE (Specialist Competence-Aware Learning), a two-stage training framework that first trains the Router with specialist correctness supervision and then optimizes its selections through reinforcement learning. The second stage uses a Performance Gain Reward (PGR) that measures how specialist information affects the generator's answer correctness relative to answering without that information. Experiments on eight text-based and multimodal medical QA benchmarks show that MedRouter outperforms the strongest routing baseline by 8% in average accuracy. Our analysis of specialist outputs further reveals distinct strengths and complementary question-level coverage, motivating learned routing to combine these capabilities for more comprehensive medical reasoning.
Abstract:Medical question answering spans diverse specialties and modalities, and individual medical large language models (LLMs) exhibit distinct strengths across tasks and domains. This heterogeneity suggests that combining specialists may enable broader coverage of medical questions than relying on any single model. However, existing LLM routing methods primarily seek to balance answer quality and inference cost, leaving open how to exploit differences in specialist competence to improve medical reasoning. In this paper, we introduce MedRouter, an agentic system that uses an embedding-based multi-label router to select and query specialist LLMs, then passes their responses to a generator to produce the final answer. We further propose SCALE (Specialist Competence-Aware Learning), a two-stage training framework that first trains the Router with specialist correctness supervision and then optimizes its selections through reinforcement learning. The second stage uses a Performance Gain Reward (PGR) that measures how specialist information affects the generator's answer correctness relative to answering without that information. Experiments on eight text-based and multimodal medical QA benchmarks show that MedRouter outperforms the strongest routing baseline by 8% in average accuracy. Our analysis of specialist outputs further reveals distinct strengths and complementary question-level coverage, motivating learned routing to combine these capabilities for more comprehensive medical reasoning.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2609.33119 [cs.CL]
(or arXiv:2609.33119v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.33119 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 299 cuvinte
Subiecte conexe
Presentation: Designing AI Platforms for Reliability: Tools for Certainty, Agents for Discovery
InfoQ · 27 mai 2026
Reducing medical claims review time with AI on AWS: The EXL Medical IDP solution
AWS · 21 sept. 2026
What are Agentic Workflows?
Databricks · 6 aug. 2026
Opaque recurrence, and other AI terms that you should probably know
TechCrunch · 7 sept. 2026