analiza aiqbench
HiDe challenges the prevailing assumption that MLLM limitations on high-resolution images stem from object size, identifying complex background interference as the primary bottleneck instead. This training-free framework leverages token-wise attention decoupling to isolate key visual regions while preserving spatial layouts, achieving state-of-the-art results on major benchmarks with 75% reduced memory usage compared to prior methods. The approach demonstrates that precise alignment with target regions, rather than raw resolution scaling, is critical for improving visual understanding in cluttered environments.
HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling
arXiv:2510.00054v3 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) have made significant strides in visual understanding tasks. However, their performance on high-resolution images remains suboptimal. While existing approaches often attribute this limitation to perceptual constraints and argue that MLLMs struggle to recognize small objects, leading them to use "zoom in" strategies for better detail, our analysis reveals a different cause: the main issue is not object size, but rather caused by complex background interference. We systematically analyze this "zoom in" operation through a series of decoupling experiments and propose the Hierarchical Decoupling Framework (HiDe), a training-free framework that uses Token-wise Attention Decoupling (TAD) to decouple the question tokens and identify the key information tokens, then leverages their attention weights to achieve precise alignment with the target visual regions. Subsequently, it employs Layout-Preserving Decoupling (LPD) to decouple these regions from the background and reconstructs a compact representation that preserves essential spatial layouts while eliminating background interference. HiDe sets a new SOTA on V*Bench, HRBench4K, and HRBench8K, boosting Qwen2.5-VL 7B and InternVL3 8B to SOTA (92.1% and 91.6% on V*Bench), even surpassing RL methods. After optimization, HiDe uses 75% less memory than the previous training-free approach. Code is provided in https://tennine2077.github.io/HiDe.github.io/.
Abstract:Multimodal Large Language Models (MLLMs) have made significant strides in visual understanding tasks. However, their performance on high-resolution images remains suboptimal. While existing approaches often attribute this limitation to perceptual constraints and argue that MLLMs struggle to recognize small objects, leading them to use "zoom in" strategies for better detail, our analysis reveals a different cause: the main issue is not object size, but rather caused by complex background interference. We systematically analyze this "zoom in" operation through a series of decoupling experiments and propose the Hierarchical Decoupling Framework (HiDe), a training-free framework that uses Token-wise Attention Decoupling (TAD) to decouple the question tokens and identify the key information tokens, then leverages their attention weights to achieve precise alignment with the target visual regions. Subsequently, it employs Layout-Preserving Decoupling (LPD) to decouple these regions from the background and reconstructs a compact representation that preserves essential spatial layouts while eliminating background interference. HiDe sets a new SOTA on V*Bench, HRBench4K, and HRBench8K, boosting Qwen2.5-VL 7B and InternVL3 8B to SOTA (92.1% and 91.6% on V*Bench), even surpassing RL methods. After optimization, HiDe uses 75% less memory than the previous training-free approach. Code is provided in this https URL.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
arXiv:2510.00054 [cs.CV]
(or arXiv:2510.00054v3 [cs.CV] for this version)
Text integral sub licența deschisă a publicației · 214 cuvinte
Subiecte conexe
Rethinking Sales Lead Scoring with LLM-based Hierarchical Preference Ranking
arXiv · 4 iun. 2026
ScaleMPA: Rethinking Scalable RRT* Acceleration With a Grid-Native Representation
arXiv · 22 sept. 2026
Rethinking the Relationship between the Power Law and Hierarchical Structures
arXiv · 6 oct. 2026
Rethinking World-Action Model for Compositional and In-Context Robotic Manipulation
arXiv · 5 oct. 2026