aiqbench analysis
This paper introduces Flow-Based Distribution Matching (FBDM), a non-adversarial self-supervised learning framework that improves training efficiency by 1.48 to 1.83 times over existing adversarial methods while maintaining competitive performance on standard benchmarks like CIFAR and ImageNet. By leveraging explicit geometric references and spherical conditional velocity regression, the approach reduces computational costs without significantly increasing GPU memory usage, offering a practical advancement for large-scale representation learning.
Learning a Flow to Self-Supervised Representations
arXiv:2609.29350v1 Announce Type: cross Abstract: Explicit geometric references offer a direct way to structure self-supervised representations. Existing adversarial distribution-matching formulations, however, require costly encoder-critic optimization. We introduce Flow-Based Distribution Matching (FBDM), a non-adversarial framework that learns this reference-directed geometry through spherical conditional velocity regression. An ETF-inspired reference allows its number of components K' to exceed the auxiliary flow dimension d* while retaining structured geometric separation. We assign both augmented views of each image to the same target, while limiting how many images each reference center can receive. An explicit alignment loss further pulls the two views' representations closer together. Experiments across benchmarks ranging from CIFAR to ImageNet show that FBDM achieves performance nearly on par with DM and remains competitive with existing SSL methods. Matched training-cost comparisons show a 1.48- to 1.83-fold speedup over DM with a negligible increase in GPU memory usage. We also provide a theoretical explanation for the usefulness of the learned representations: under stated conditions, we bound the downstream misclassification rate in terms of the FBDM pretraining loss.
Abstract:Explicit geometric references offer a direct way to structure self-supervised representations. Existing adversarial distribution-matching formulations, however, require costly encoder-critic optimization. We introduce Flow-Based Distribution Matching (FBDM), a non-adversarial framework that learns this reference-directed geometry through spherical conditional velocity regression. An ETF-inspired reference allows its number of components K' to exceed the auxiliary flow dimension d* while retaining structured geometric separation. We assign both augmented views of each image to the same target, while limiting how many images each reference center can receive. An explicit alignment loss further pulls the two views' representations closer together. Experiments across benchmarks ranging from CIFAR to ImageNet show that FBDM achieves performance nearly on par with DM and remains competitive with existing SSL methods. Matched training-cost comparisons show a 1.48- to 1.83-fold speedup over DM with a negligible increase in GPU memory usage. We also provide a theoretical explanation for the usefulness of the learned representations: under stated conditions, we bound the downstream misclassification rate in terms of the FBDM pretraining loss.
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Machine Learning (stat.ML)
arXiv:2609.29350 [cs.CV]
(or arXiv:2609.29350v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2609.29350 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 270 words
Related coverage
From Pixels, Without Pre-training: Joint Generative and Self-Supervised Representation Learning in One Model
arXiv · Oct 10, 2026
Timestamp-Aware Spatio-Temporal Graph Contrastive Learning for Network Intrusion Detection
arXiv · Jun 17, 2026
Meta-Representational Predictive Coding: Neuroscience-Informed Self-Supervised Learning
arXiv · Sep 22, 2026
Broaden Your Views for Self-Supervised Video Learning
arXiv · Oct 6, 2026