aiqbench analysis
VenusGenome introduces a novel masked autoencoder architecture specifically designed to learn microbial genome representations by accounting for biological context rather than just assembly artifacts. By pretraining on over 1.7 million assemblies, the model achieves state-of-the-art performance in phenotype prediction and successfully links representation sensitivity to experimental fitness defects. This work advances machine learning applications in bioinformatics by demonstrating that high-ratio masking combined with genome-aware attention yields superior gene-resolution embeddings compared to standard BERT-style approaches.
VenusGenome: Genome-Aware Masked Autoencoding for Microbial Genome Representation Learning
arXiv:2611.05405v1 Announce Type: cross Abstract: Genomic-context models have shown that contextualizing protein embeddings can improve predictions of microbial function. We observe that a gene's protein embedding is partially predictable from the embeddings of neighboring genes in the genome, motivating high-ratio masking that requires reconstruction from sparser genomic context. Genomic modeling must also distinguish biological relationships from assembly conventions, since the ordering of separate contigs does not imply adjacency and strand labels depend on the reference orientation. We therefore introduce VenusGenome, a gene-resolution masked autoencoder that combines visible-only encoding with genome-aware attention incorporating contig boundaries, relative strand orientations, and gene positions. We pretrain this model on a curated corpus of over 1.7 million microbial genome assemblies from isolate and metagenomic sources, using frozen protein embeddings as gene-level inputs. Across genome-level phenotype prediction and gene-level tasks, the resulting frozen representations achieve the highest mean performance among the evaluated models on nine of eleven primary metrics. An exploratory gene-input perturbation analysis further links representation sensitivity to experimentally measured fitness defects. Ablations support the contribution of genome-aware attention and show the benefits of masked autoencoding over BERT-style masked modeling, with further gains from higher masking ratios.
Abstract:Genomic-context models have shown that contextualizing protein embeddings can improve predictions of microbial function. We observe that a gene's protein embedding is partially predictable from the embeddings of neighboring genes in the genome, motivating high-ratio masking that requires reconstruction from sparser genomic context. Genomic modeling must also distinguish biological relationships from assembly conventions, since the ordering of separate contigs does not imply adjacency and strand labels depend on the reference orientation. We therefore introduce VenusGenome, a gene-resolution masked autoencoder that combines visible-only encoding with genome-aware attention incorporating contig boundaries, relative strand orientations, and gene positions. We pretrain this model on a curated corpus of over 1.7 million microbial genome assemblies from isolate and metagenomic sources, using frozen protein embeddings as gene-level inputs. Across genome-level phenotype prediction and gene-level tasks, the resulting frozen representations achieve the highest mean performance among the evaluated models on nine of eleven primary metrics. An exploratory gene-input perturbation analysis further links representation sensitivity to experimentally measured fitness defects. Ablations support the contribution of genome-aware attention and show the benefits of masked autoencoding over BERT-style masked modeling, with further gains from higher masking ratios.
Genomics (q-bio.GN); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2611.05405 [q-bio.GN]
(or arXiv:2611.05405v1 [q-bio.GN] for this version)
https://doi.org/10.48550/arXiv.2611.05405 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 286 words
Related coverage
Distilling Drifting Transformers with Representation Autoencoders
arXiv · Jun 16, 2026
Learning Disentangled Representations with Quantum Variational Autoencoders
arXiv · Oct 7, 2026
RePAIR: Predictive Self-Supervised Representation Learning in Chess
arXiv · Jun 12, 2026
Spectral characteristics of autoencoder parameters as a vector representation of data
arXiv · Sep 4, 2026