analiza aiqbench
This research introduces MoGI, a novel method for measuring the influence of specific text and image inputs on predictions within multimodal attributed graphs. By analyzing how replacing node representations affects both local and nearby nodes, the study demonstrates that distributed responses capture critical information missed by traditional attribution techniques without requiring model retraining. The proposed framework offers a more robust way to evaluate reliance on different data modalities in graph-based machine learning systems.
Measuring the Influence of Modality-Specific Node Representations on Graphs
arXiv:2610.23863v1 Announce Type: cross Abstract: Multimodal attributed graphs combine graph structure with node text and image representations in recommendation and multimedia analysis. Graph predictors propagate these representations across connected nodes, so one input can change predictions at its own and nearby nodes. Measuring this distributed response reveals which modality-specific inputs a trained model relies on and how predictor design changes that reliance. We study influence measurement for modality-specific node representations. Given an encoded graph, a trained predictor, a training split, and fixed candidate representations, the task outputs one influence score per candidate and separate text and image rankings. We score a candidate by replacing it with its modality's training-node mean and comparing the original and changed predictions. Existing feature attribution and data valuation generally target one output, training examples, whole nodes, or graph elements rather than prediction changes across graph nodes. Our empirical study shows that nearby-node responses contain information missed by source-only scores and simple proxies, while response magnitude alone does not identify which class changes support the task. Guided by these observations, we propose MoGI (Modality-specific Graph Influence). It propagates training labels to define task direction, reads the same replacement response for the favored class and full class distribution, and averages the resulting ranks. MoGI requires no predictor retraining or learned scoring parameters. Under a shared five-dataset, eight-predictor benchmark, it achieves the largest aggregate held-out loss increase against all five external methods at every tested ranking cutoff.
Abstract:Multimodal attributed graphs combine graph structure with node text and image representations in recommendation and multimedia analysis. Graph predictors propagate these representations across connected nodes, so one input can change predictions at its own and nearby nodes. Measuring this distributed response reveals which modality-specific inputs a trained model relies on and how predictor design changes that reliance. We study influence measurement for modality-specific node representations. Given an encoded graph, a trained predictor, a training split, and fixed candidate representations, the task outputs one influence score per candidate and separate text and image rankings. We score a candidate by replacing it with its modality's training-node mean and comparing the original and changed predictions. Existing feature attribution and data valuation generally target one output, training examples, whole nodes, or graph elements rather than prediction changes across graph nodes. Our empirical study shows that nearby-node responses contain information missed by source-only scores and simple proxies, while response magnitude alone does not identify which class changes support the task. Guided by these observations, we propose MoGI (Modality-specific Graph Influence). It propagates training labels to define task direction, reads the same replacement response for the favored class and full class distribution, and averages the resulting ranks. MoGI requires no predictor retraining or learned scoring parameters. Under a shared five-dataset, eight-predictor benchmark, it achieves the largest aggregate held-out loss increase against all five external methods at every tested ranking cutoff.
Social and Information Networks (cs.SI); Machine Learning (cs.LG); Multimedia (cs.MM)
arXiv:2610.23863 [cs.SI]
(or arXiv:2610.23863v1 [cs.SI] for this version)
https://doi.org/10.48550/arXiv.2610.23863 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 333 cuvinte