analiza aiqbench
This research introduces a topology-conditioned steering mechanism that uses persistent homology descriptors to analyze and guide Large Language Models in handling ill-posed questions like those involving ambiguity or conflicting statements. By treating internal hidden states as point clouds, the method extracts compact signals to modify model activations during prefill and decoding, effectively directing the model toward appropriate responses such as abstention or clarification. The approach demonstrates improved performance across multiple open-weight models, offering a novel way to probe and control reasoning processes without relying on black-box input-output analysis.
TIPS: Topological Ill-Posedness Probing and Steering in Large Language Models
arXiv:2606.23590v2 Announce Type: replace Abstract: Ill-posed questions, including those involving ambiguity, under-specification, or conflicting statements, may admit no valid answer or multiple plausible answers, posing a significant challenge for large language models (LLMs) despite their otherwise strong performance. Existing work often treats LLM reasoning as a black box and focuses on input-output analysis. Can a compact, unified topological representation of internal model states capture diverse sources of ill-posedness and steer reasoning toward responses appropriate to each source? To this end, we study the internal state from a topological perspective by treating the contextual hidden states of its prompt tokens at a single transformer layer as a point cloud, capturing the input problem's relational structure as encoded by the model. Our analysis shows that zero- and one-dimensional persistent homology---tracking how connected components merge as the distance threshold increases ($H_0$) and the emergence and filling of one-dimensional loops in the filtration ($H_1$), respectively---yield seven compact descriptors of ill-posedness that also serve as control signals for steering model reasoning. Thus, we develop a topology-conditioned steering mechanism. It retrieves topologically similar positive and negative examples for each query to form a local activation-space contrast. The resulting query-specific direction modifies internal states during prefill and decoding, guiding the model's responses toward suitable abstention or clarification tailored to the underlying source of ill-posedness. Empirical evaluations on two datasets and open-weight models spanning four families demonstrate the effectiveness of this compact representation for ill-posedness classification, while topology-conditioning significantly improved steering performance.
Abstract:Ill-posed questions, including those involving ambiguity, under-specification, or conflicting statements, may admit no valid answer or multiple plausible answers, posing a significant challenge for large language models (LLMs) despite their otherwise strong performance. Existing work often treats LLM reasoning as a black box and focuses on input-output analysis. Can a compact, unified topological representation of internal model states capture diverse sources of ill-posedness and steer reasoning toward responses appropriate to each source? To this end, we study the internal state from a topological perspective by treating the contextual hidden states of its prompt tokens at a single transformer layer as a point cloud, capturing the input problem's relational structure as encoded by the model. Our analysis shows that zero- and one-dimensional persistent homology---tracking how connected components merge as the distance threshold increases ($H_0$) and the emergence and filling of one-dimensional loops in the filtration ($H_1$), respectively---yield seven compact descriptors of ill-posedness that also serve as control signals for steering model reasoning. Thus, we develop a topology-conditioned steering mechanism. It retrieves topologically similar positive and negative examples for each query to form a local activation-space contrast. The resulting query-specific direction modifies internal states during prefill and decoding, guiding the model's responses toward suitable abstention or clarification tailored to the underlying source of ill-posedness. Empirical evaluations on two datasets and open-weight models spanning four families demonstrate the effectiveness of this compact representation for ill-posedness classification, while topology-conditioning significantly improved steering performance.
Artificial Intelligence (cs.AI)
arXiv:2606.23590 [cs.AI]
(or arXiv:2606.23590v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.23590 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 331 cuvinte
Subiecte conexe
Benchtalks #2: The future of coding benchmarks
Snorkel AI Blog · 3 iun. 2026
"I am an AI Safety Researcher"
LessWrong · 23 sept. 2026
Would We See It Coming? Preference Falsification Cascades in Multi-Agent Systems
LessWrong · 4 aug. 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · 8 oct. 2026