aiqbench analysis
This study audits deep video models estimating left-ventricular ejection fraction, revealing a critical gap where spatial attribution is strong but temporal reliance on decisive frames (end-systolic and end-diastolic) remains weak. The findings demonstrate that architectural choices fundamentally determine this temporal blindness rather than training convergence or accuracy improvements, challenging the assumption that anatomical faithfulness implies faithful reasoning over time. For developers of medical AI vision systems, these results highlight a necessary focus on explicit temporal grounding mechanisms to ensure models truly understand dynamic physiological processes.
Anatomically Faithful but Temporally Diffuse: Auditing Attribution for Left-Ventricular Ejection-Fraction Estimation from Echocardiography
arXiv:2607.13738v3 Announce Type: replace-cross Abstract: Deep video models estimate left-ventricular ejection fraction (EF) from echocardiography with near-expert accuracy, and post-hoc attribution is increasingly used to certify that such models look at the right place. Because EF is defined by the end-systolic (ES) and end-diastolic (ED) frames, a faithful explanation must localize not only the left ventricle in space but also the decisive frames in time. We audit attribution faithfulness along three axes -- spatial grounding, perturbation, and temporal reliance -- for two architecturally distinct regressors fine-tuned on EchoNet-Dynamic: a self-supervised VideoMAE transformer audited with Chefer relevance propagation, and a Kinetics-pretrained R(2+1)D convolutional network audited with Grad-CAM. All statistics are reported with bootstrap 95% confidence intervals over the full 1,276-study test split, against explicit chance baselines and attention-rollout and random-attribution controls. We find a pronounced asymmetry rather than a temporal null. Both models are strongly anatomically faithful (3.04x and 3.76x chance) but only weakly localized in time: tubelet occlusion places causal reliance on ES/ED at 1.05x chance for VideoMAE (95% CI [1.037, 1.068]) and 1.15x for R(2+1)D ([1.134, 1.166]) -- a threefold weaker concentration than in space -- and the attribution faithfully mirrors this weakness. Separating convergence from architecture shows that temporal reliance is architecture-determined and convergence-invariant: R(2+1)D attains 1.14x after a single training epoch and 1.15x at convergence, while both VideoMAE variants stay at 1.05x regardless of pretraining or accuracy. Spatial faithfulness therefore does not imply temporal faithfulness, and the shortfall is not remedied by further training.
Abstract:Deep video models estimate left-ventricular ejection fraction (EF) from echocardiography with near-expert accuracy, and post-hoc attribution is increasingly used to certify that such models look at the right place. Because EF is defined by the end-systolic (ES) and end-diastolic (ED) frames, a faithful explanation must localize not only the left ventricle in space but also the decisive frames in time. We audit attribution faithfulness along three axes -- spatial grounding, perturbation, and temporal reliance -- for two architecturally distinct regressors fine-tuned on EchoNet-Dynamic: a self-supervised VideoMAE transformer audited with Chefer relevance propagation, and a Kinetics-pretrained R(2+1)D convolutional network audited with Grad-CAM. All statistics are reported with bootstrap 95% confidence intervals over the full 1,276-study test split, against explicit chance baselines and attention-rollout and random-attribution controls. We find a pronounced asymmetry rather than a temporal null. Both models are strongly anatomically faithful (3.04x and 3.76x chance) but only weakly localized in time: tubelet occlusion places causal reliance on ES/ED at 1.05x chance for VideoMAE (95% CI [1.037, 1.068]) and 1.15x for R(2+1)D ([1.134, 1.166]) -- a threefold weaker concentration than in space -- and the attribution faithfully mirrors this weakness. Separating convergence from architecture shows that temporal reliance is architecture-determined and convergence-invariant: R(2+1)D attains 1.14x after a single training epoch and 1.15x at convergence, while both VideoMAE variants stay at 1.05x regardless of pretraining or accuracy. Spatial faithfulness therefore does not imply temporal faithfulness, and the shortfall is not remedied by further training.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
arXiv:2607.13738 [cs.CV]
(or arXiv:2607.13738v3 [cs.CV] for this version)
Full text under the publisher's open licence · 259 words
Related coverage
Projection sampling preserves capabilities but diminishes gains from alignment fine-tuning
LessWrong · Oct 16, 2026
Orthogonal Ensembles and Tested Explanations for Performer-Independent Body-Motion Emotion Recognition
arXiv · Sep 3, 2026
A Systematic Benchmark of Explainable Methods for Temporal Attribution in Sequential Recommendation Systems
arXiv · Sep 24, 2026
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · Oct 8, 2026