aiqbench analysis
This research addresses a critical bottleneck in multi-hop question answering by introducing PhaseGraph, a method that calibrates heterogeneous retrieval scores from vector and graph sources onto a unified scale. By utilizing percentile-rank normalization, the approach achieves stable fusion without discarding magnitude information, yielding measurable gains in last-hop accuracy on MuSiQue and 2WikiMultiHopQA benchmarks. The study highlights that score commensuration is a robust design choice, with specific calibration operators proving more resilient than traditional min-max normalization across varying test splits.
Calibrated Fusion for Heterogeneous Graph-Vector Retrieval in Multi-Hop QA
arXiv:2603.28886v3 Announce Type: replace-cross Abstract: Graph-augmented retrieval combines dense similarity with graph-based relevance signals such as Personalized PageRank (PPR), but these scores have different distributions and are not directly comparable. We study this as a score calibration problem for heterogeneous retrieval fusion in multi-hop question answering. Our method, PhaseGraph, maps vector and graph scores to a common unit-free scale using percentile-rank normalization (PIT) before fusion, enabling stable combination without discarding magnitude information. Across MuSiQue and 2WikiMultiHopQA, calibrated fusion improves held-out last-hop retrieval on HippoRAG2-style benchmarks: LastHop@10 increases from 69.1% to 71.0% on MuSiQue (15W/5L, p=0.041, n=514) and LastHop@5 from 51.7% to 53.6% on 2WikiMultiHopQA (11W/2L, p=0.023, n=491), both on independent held-out test splits. Against the official HippoRAG 2 pipeline on 2WikiMultiHopQA, calibrated fusion is ahead at LastHop@10 (+6.3pp, p<10^-3) and behind at LastHop@5 (-8.4pp), a cutoff-dependent cross-over we report in full. A theory-driven ablation shows that percentile-based calibration is directionally more robust than min-max normalization on both tune and test splits (1W/6L, p=0.125), while Boltzmann weighting performs comparably to linear fusion after calibration (0W/3L, p=0.25). These results suggest that score commensuration is a robust design choice, and the exact post-calibration operator appears to matter less on these benchmarks.
Abstract:Graph-augmented retrieval combines dense similarity with graph-based relevance signals such as Personalized PageRank (PPR), but these scores have different distributions and are not directly comparable. We study this as a score calibration problem for heterogeneous retrieval fusion in multi-hop question answering. Our method, PhaseGraph, maps vector and graph scores to a common unit-free scale using percentile-rank normalization (PIT) before fusion, enabling stable combination without discarding magnitude information. Across MuSiQue and 2WikiMultiHopQA, calibrated fusion improves held-out last-hop retrieval on HippoRAG2-style benchmarks: LastHop@10 increases from 69.1% to 71.0% on MuSiQue (15W/5L, p=0.041, n=514) and LastHop@5 from 51.7% to 53.6% on 2WikiMultiHopQA (11W/2L, p=0.023, n=491), both on independent held-out test splits. Against the official HippoRAG 2 pipeline on 2WikiMultiHopQA, calibrated fusion is ahead at LastHop@10 (+6.3pp, p<10^-3) and behind at LastHop@5 (-8.4pp), a cutoff-dependent cross-over we report in full. A theory-driven ablation shows that percentile-based calibration is directionally more robust than min-max normalization on both tune and test splits (1W/6L, p=0.125), while Boltzmann weighting performs comparably to linear fusion after calibration (0W/3L, p=0.25). These results suggest that score commensuration is a robust design choice, and the exact post-calibration operator appears to matter less on these benchmarks.
v4: MuSiQue LastHop recomputed against the terminal-hop passage (v1-v3 scored the last supporting paragraph in MuSiQue's paragraph order): vector-only 69.1 -> PhaseGraph 71.0 at @10 (15W/5L, p=.041); True RRF 71.8 (25W/11L, p=.029); no paired advantage over RRF on either benchmark; embedding-realization sensitivity disclosed. 10 pages, 6 figures, 9 tables
Information Retrieval (cs.IR); Machine Learning (cs.LG)
arXiv:2603.28886 [cs.IR]
(or arXiv:2603.28886v3 [cs.IR] for this version)
Full text under the publisher's open licence · 256 words
Related coverage
From RAG to Agentic AI: Building the Next Generation of Intelligent Enterprise Systems
KDnuggets · Sep 8, 2026
GraphRAG with TypeSafe Jev: A System One Approach to Scalable Knowledge Graphs
Towards Data Science · Sep 27, 2026
Hybrid Retrieval-Augmented Generation with Knowledge Graph Expansion, RRF Fusion, and Per-Chunk Grounded Evaluation for Enterprise Document Search
arXiv · Sep 3, 2026
MSGAT: Multi-Head Spiking Graph Attention with Similarity-Space Fusion for Image-Text Retrieval
arXiv · Oct 10, 2026