aiqbench analysis
BioM-JEPA advances self-supervised learning in single-cell biology by shifting prediction targets from individual genes to graph-connected blocks, resulting in embeddings with higher effective rank and improved throughput via linear attention. This architecture outperforms standard reconstruction baselines on CellBench tasks while maintaining robust biological signal across perturbation-response evaluations. The approach offers a scalable framework for modeling coordinated gene programs without relying on quadratic complexity matrices common in current genomic AI models.
BioM-JEPA: joint-embedding prediction of graph-connected gene blocks in single cells
arXiv:2608.05928v1 Announce Type: new Abstract: Single-cell transcriptomes are sparse observations of coordinated biological programmes, yet most self-supervised models learn by reconstructing individual genes. Here we present BioM-JEPA, a joint-embedding predictive architecture that instead predicts aggregate representations of graph-connected gene blocks defined by protein-association and corpus-derived coexpression evidence. A student network infers each target-block representation from the remaining genes in a cell, while a slowly updated teacher supplies the corresponding target from the full observed gene set. Under the reported extraction procedure, block-level prediction produced embeddings with higher effective rank and weaker association with detected-gene depth in the tested diagnostics than token-prediction, random-block and reconstruction controls. Across CellBench tasks, frozen BioM-JEPA embeddings retained expression, pathway and neighbourhood information and achieved the lowest aggregate perturbation-response error among the evaluated models. Representation diagnostics were also consistent with canonical pancreatic programmes and compositional relationships between genetic perturbations. Linear attention avoids constructing a quadratic gene-by-gene attention matrix; in a matched one-epoch hPancreas experiment at batch size 8, BioM-JEPA provided 5.75-fold higher fine-tuning throughput and 3.76-fold higher held-out embedding throughput than scFoundation. Together, these results support graph-connected gene blocks as useful prediction units for JEPA-style representation learning in single-cell biology.
Abstract:Single-cell transcriptomes are sparse observations of coordinated biological programmes, yet most self-supervised models learn by reconstructing individual genes. Here we present BioM-JEPA, a joint-embedding predictive architecture that instead predicts aggregate representations of graph-connected gene blocks defined by protein-association and corpus-derived coexpression evidence. A student network infers each target-block representation from the remaining genes in a cell, while a slowly updated teacher supplies the corresponding target from the full observed gene set. Under the reported extraction procedure, block-level prediction produced embeddings with higher effective rank and weaker association with detected-gene depth in the tested diagnostics than token-prediction, random-block and reconstruction controls. Across CellBench tasks, frozen BioM-JEPA embeddings retained expression, pathway and neighbourhood information and achieved the lowest aggregate perturbation-response error among the evaluated models. Representation diagnostics were also consistent with canonical pancreatic programmes and compositional relationships between genetic perturbations. Linear attention avoids constructing a quadratic gene-by-gene attention matrix; in a matched one-epoch hPancreas experiment at batch size 8, BioM-JEPA provided 5.75-fold higher fine-tuning throughput and 3.76-fold higher held-out embedding throughput than scFoundation. Together, these results support graph-connected gene blocks as useful prediction units for JEPA-style representation learning in single-cell biology.
34 pages, 6 figures, and 13 supplementary tables (Tables S1-S13); includes Supplementary Information with detailed training and evaluation protocols. Numerical source data for all figures are provided as ancillary files; training code and the BioM-JEPA checkpoint will be released via GitHub
Machine Learning (cs.LG)
arXiv:2608.05928 [cs.LG]
(or arXiv:2608.05928v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.05928 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 248 words
Related coverage
Phenomenon-Graph JEPA: Label-Efficient Representation Learning for Contactless Cardiorespiratory Sensing
arXiv · Sep 30, 2026
A Generalization Theory for JEPA-Based World Models
arXiv · Sep 30, 2026
BeliefGraph-JEPA: Structured Latent World Models for Action-Conditioned Time Series
arXiv · Oct 7, 2026
Metro-WM: Long-Horizon Latent Planning with Realisable Sub-Goals
arXiv · Sep 29, 2026