aiqbench analysis
This research investigates the internal representation of texture evidence within frozen Segment Anything Models, determining whether failures in material-based segmentation stem from missing data or misaligned readouts. By analyzing multiscale features and proposal banks without fine-tuning, the study reveals that coarse-level textures are preserved but require specific assembly mechanisms rather than simple model adaptation. This analysis provides critical insights for developers seeking to extend foundation models into texture-heavy domains like robotics or industrial inspection where object identity is secondary to material properties.
Where Does Texture Evidence Live in SAM? Features, Proposal Masks, and Texture Segmentation
arXiv:2606.14755v1 Announce Type: cross Abstract: Texture segmentation stresses foundation segmentation because meaningful regions are defined by material or repeated appearance rather than object identity. Segment Anything Models (SAMs) often fail by default on such texture-defined partitions, but this failure is ambiguous: the texture evidence may be absent, missing from the proposal bank, or present but selected or assembled incorrectly by an object-centric readout. We ask what texture-relevant evidence is already preserved in frozen SAM before adaptation. We study two frozen evidence spaces: multiscale features, probed with a minimal clustering readout, and the automatic proposal bank, treated as evidence for a supervised consolidation readout. SAM is frozen throughout; we do not fine-tune the backbone or retrain the proposal generator. Across RWTD, STLD, an ADE20K-selected refined-crop complement, and a ControlNet-stitched PTD bridge archive, frozen SAM is not a texture segmenter by default, but its failures are not simple texture blindness. Coarse frozen features preserve texture organization, and proposal banks often contain texture-aligned masks or fragments. Natural scenes more often require assembly and commitment over fragments, while cleaner synthetic cases more often reduce to selecting an already coherent proposal. Default mask failure should therefore be decomposed into representation evidence, proposal-bank support, readout mismatch, and commitment failure.
Title:Where Does Texture Evidence Live in SAM? Features, Proposal Masks, and Texture Segmentation
Abstract:Texture segmentation stresses foundation segmentation because meaningful regions are defined by material or repeated appearance rather than object identity. Segment Anything Models (SAMs) often fail by default on such texture-defined partitions, but this failure is ambiguous: the texture evidence may be absent, missing from the proposal bank, or present but selected or assembled incorrectly by an object-centric readout. We ask what texture-relevant evidence is already preserved in frozen SAM before adaptation. We study two frozen evidence spaces: multiscale features, probed with a minimal clustering readout, and the automatic proposal bank, treated as evidence for a supervised consolidation readout. SAM is frozen throughout; we do not fine-tune the backbone or retrain the proposal generator. Across RWTD, STLD, an ADE20K-selected refined-crop complement, and a ControlNet-stitched PTD bridge archive, frozen SAM is not a texture segmenter by default, but its failures are not simple texture blindness. Coarse frozen features preserve texture organization, and proposal banks often contain texture-aligned masks or fragments. Natural scenes more often require assembly and commitment over fragments, while cleaner synthetic cases more often reduce to selecting an already coherent proposal. Default mask failure should therefore be decomposed into representation evidence, proposal-bank support, readout mismatch, and commitment failure.
26 pages, 13 figures, 20 tables. Code available at this https URL
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
I.4.6; I.5.3; I.2.10
arXiv:2606.14755 [cs.CV]
(or arXiv:2606.14755v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2606.14755 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 324 words
Related coverage
IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question Answering
arXiv · Aug 18, 2026
UltraG-Bench: A Multi-task Benchmark for assessing Large Vision-Language Models on Pixel-level Evidence Grounding in Ultrasound
arXiv · Sep 28, 2026
HERO: Hierarchical Evidential Reasoning Optimization for Radiology Report Generation via Reason-then-Summarize
arXiv · Aug 5, 2026
Multi-Agent Self-Improving Reinforcement Learning for Video Reasoning
arXiv · Sep 1, 2026