analiza aiqbench
QA-Merging introduces an activation-based framework that dynamically selects transformer layers for calibration, enabling large reasoning models to switch between short and long chain-of-thought behaviors without retraining. By focusing only on layers where Long-CoT and Short-CoT patterns diverge most significantly, this method reduces inference latency while maintaining high accuracy across complex benchmarks. The approach represents a practical step toward efficient adaptive reasoning in next-generation AI systems.
QA-Merging: Query-Adaptive Reasoning via Layer Selective Model Merging
arXiv:2601.03506v2 Announce Type: replace Abstract: Recent large reasoning models (LRMs) have achieved strong performance on complex reasoning tasks by generating a long chain-of-thought (Long-CoT). However, such lengthy reasoning is often unnecessary for simple queries, leading to additional computation and latency. Existing approaches to adaptive reasoning typically rely on retraining the model or designing sophisticated prompting, which are either prohibitively expensive or highly sensitive to the prompt formulation. Model merging provides a more balanced alternative for adaptive reasoning by avoiding expensive training and integrating Long-CoT and Short-CoT behaviors. However, existing merging methods are often static and input-agnostic, or rely on costly all-layer calibration, which limits their effectiveness for query-adaptive reasoning. To tackle these challenges, we propose Query-adaptive Layer Selective Merging (QA-Merging), an activation-based merging framework that integrates a Long-CoT model and a Short-CoT model to obtain a query-adaptive reasoner without training from scratch or requiring large-scale additional data. QA-Merging first constructs a small pattern-labeled calibration set that assigns each query an appropriate reasoning pattern. Motivated by our empirical analysis that Long-CoT and Short-CoT behaviors diverge unevenly across transformer layers, QA-Merging identifies layers with high reasoning pattern divergence and calibrates only these layers through feature alignment and contrastive shaping, while applying closed-form hidden-state correction to the remaining layers. Experiments on seven widely used reasoning benchmarks across two model scales demonstrate that QA-Merging reduces inference cost and maintains strong performance.
Abstract:Recent large reasoning models (LRMs) have achieved strong performance on complex reasoning tasks by generating a long chain-of-thought (Long-CoT). However, such lengthy reasoning is often unnecessary for simple queries, leading to additional computation and latency. Existing approaches to adaptive reasoning typically rely on retraining the model or designing sophisticated prompting, which are either prohibitively expensive or highly sensitive to the prompt formulation. Model merging provides a more balanced alternative for adaptive reasoning by avoiding expensive training and integrating Long-CoT and Short-CoT behaviors. However, existing merging methods are often static and input-agnostic, or rely on costly all-layer calibration, which limits their effectiveness for query-adaptive reasoning. To tackle these challenges, we propose Query-adaptive Layer Selective Merging (QA-Merging), an activation-based merging framework that integrates a Long-CoT model and a Short-CoT model to obtain a query-adaptive reasoner without training from scratch or requiring large-scale additional data. QA-Merging first constructs a small pattern-labeled calibration set that assigns each query an appropriate reasoning pattern. Motivated by our empirical analysis that Long-CoT and Short-CoT behaviors diverge unevenly across transformer layers, QA-Merging identifies layers with high reasoning pattern divergence and calibrates only these layers through feature alignment and contrastive shaping, while applying closed-form hidden-state correction to the remaining layers. Experiments on seven widely used reasoning benchmarks across two model scales demonstrate that QA-Merging reduces inference cost and maintains strong performance.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2601.03506 [cs.CL]
(or arXiv:2601.03506v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2601.03506 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 318 cuvinte
Subiecte conexe
AgentOps: Operationalize agentic AI at scale with Amazon Bedrock AgentCore
AWS · 1 iun. 2026
Can we change an AI model’s architecture without retraining it from scratch?
Amazon Science · 12 oct. 2026
Optimizing ATOM and vLLM-ATOM for High-Interactivity Inference
AMD · 1 sept. 2026
Welcome Beam: a 501B open model you can own, customize, and scale
Hugging Face · 16 oct. 2026