aiqbench analysis
This paper proposes a continual training add-on for genomic foundation models that adds a latent-space prediction loss, based on Joint-Embedding Predictive Architectures, alongside standard masked or next-token objectives. Its main claims are gains on 17 genomic benchmarks under linear probing and zero-shot evaluation, plus a reported state of the art under linear probing. Readers should weigh these as benchmark results from a single research group, and the model-agnostic framing is more useful to watch than the headline numbers until independent replications appear.
JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures
arXiv:2602.17162v4 Announce Type: replace Abstract: Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature". While effective at capturing local syntax, these generative paradigms prioritize token-level reconstruction over high-level functional context. We introduce JEPA-DNA, a model-agnostic continual training framework that integrates a Joint-Embedding Predictive Architecture (JEPA) with traditional generative objectives. By supervising global sequence embeddings in a latent space, JEPA-DNA forces models to predict the functional representations of masked genomic segments, shifting the learning signal from token recovery to semantic alignment. We evaluate JEPA-DNA on 17 diverse genomic benchmark tasks, demonstrating consistent gains in linear probing and zero-shot performance regardless of the underlying GFM architecture or generative objective, and establishing a new state-of-the-art for GFMs under the linear-probing protocol. Through extensive ablation studies, we further characterize the synergistic interplay between generative and latent objectives. Our code is provided in the supplementary material.
Authors:Ariel Larey, Elay Dahan, Amit Bleiweiss, Raizy Kellerman, Guy Leib, Omri Nayshool, Dan Ofer, Tal Zinger, Dan Dominissini, Gideon Rechavi, Nicole Bussola, Simon Lee, Shane O'Connell, Dung Hoang, Marissa Wirth, Alexander W. Charney, Nati Daniel, Yoli Shavit
Abstract:Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature". While effective at capturing local syntax, these generative paradigms prioritize token-level reconstruction over high-level functional context. We introduce JEPA-DNA, a model-agnostic continual training framework that integrates a Joint-Embedding Predictive Architecture (JEPA) with traditional generative objectives. By supervising global sequence embeddings in a latent space, JEPA-DNA forces models to predict the functional representations of masked genomic segments, shifting the learning signal from token recovery to semantic alignment. We evaluate JEPA-DNA on 17 diverse genomic benchmark tasks, demonstrating consistent gains in linear probing and zero-shot performance regardless of the underlying GFM architecture or generative objective, and establishing a new state-of-the-art for GFMs under the linear-probing protocol. Through extensive ablation studies, we further characterize the synergistic interplay between generative and latent objectives. Our code is provided in the supplementary material.
Artificial Intelligence (cs.AI); Genomics (q-bio.GN)
arXiv:2602.17162 [cs.AI]
(or arXiv:2602.17162v4 [cs.AI] for this version)
Full text under the publisher's open licence · 198 words
Related coverage
Biomedical Imaging's Real Bottleneck Is the Data, Not the Model
Databricks · Oct 8, 2026
Clin-JEPA: A Multi-Phase Co-Training Framework for Joint-Embedding Predictive Pretraining on EHR Patient Trajectories
arXiv · Sep 18, 2026
World Models and Yann LeCun’s Bet Against Large Language Models
Quantum Zeitgeist · Jul 17, 2026
W-JEPA: Wavelet Joint-Embedding Predictive Architecture for Wearable Activity Recognition
arXiv · Oct 15, 2026