aiqbench analysis
This paper proposes a method for personalized gaze estimation that uses a diffusion transformer to generate augmented training images from few calibration samples, reporting a 4.1% generator-level gain over GazeGaussian on ETH-XGaze and a 0.73-degree subject-level improvement at nine calibration images. The results are self-reported on the authors' own expanded protocol and haven't been independently reproduced, so the key question for readers is whether the gains hold across other datasets and setups. For ML practitioners, the more transferable idea is conditioning a latent diffusion model on 3D scene geometry to densify sparse supervision.
DiTGaze: Geometry-Conditioned Diffusion Augmentation for Few-Shot Gaze Estimation
arXiv:2511.11231v2 Announce Type: replace-cross Abstract: Accurate person-specific gaze estimation benefits from labeled user data, but collecting images that densely cover gaze directions is burdensome. We present DiTGaze, a calibration-efficient augmentation system that follows a two-stream 3D Gaussian Splatting backbone and reformulates its convolutional rendering stage as a geometry-conditioned latent Diffusion Transformer (DiT). A learnable projection and frozen VAE bridge rasterized Gaussian features to latent tokens, while timestep and identity/expression conditioning are integrated through Adaptive Layer Normalization. Intermediate yaw-pitch sampling densifies gaze supervision, and an orthogonality constraint reduces interference between gaze, pose, and expression. On ETH-XGaze, DiTGaze reduces generator-level gaze error by 4.1 percent relative to GazeGaussian. In our runs of the expanded downstream protocol, it ranks first at every calibration budget from one to nine real images; the repeated primary comparison at N = 9 reduces subject-level estimator error by 0.73 degrees and improves 14 of 15 identities (p < 0.001). These results connect the proposed generator changes to statistically supported few-shot personalization gains.
Abstract:Accurate person-specific gaze estimation benefits from labeled user data, but collecting images that densely cover gaze directions is burdensome. We present DiTGaze, a calibration-efficient augmentation system that follows a two-stream 3D Gaussian Splatting backbone and reformulates its convolutional rendering stage as a geometry-conditioned latent Diffusion Transformer (DiT). A learnable projection and frozen VAE bridge rasterized Gaussian features to latent tokens, while timestep and identity/expression conditioning are integrated through Adaptive Layer Normalization. Intermediate yaw-pitch sampling densifies gaze supervision, and an orthogonality constraint reduces interference between gaze, pose, and expression. On ETH-XGaze, DiTGaze reduces generator-level gaze error by 4.1 percent relative to GazeGaussian. In our runs of the expanded downstream protocol, it ranks first at every calibration budget from one to nine real images; the repeated primary comparison at N = 9 reduces subject-level estimator error by 0.73 degrees and improves 14 of 15 identities (p < 0.001). These results connect the proposed generator changes to statistically supported few-shot personalization gains.
Accepted at WACV 2027. Updated and extended version of the original 2025 preprint, including revised methodology and expanded few-shot gaze-estimation experiments
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
arXiv:2511.11231 [cs.CV]
(or arXiv:2511.11231v2 [cs.CV] for this version)
Full text under the publisher's open licence · 197 words