aiqbench analysis
This research introduces PUMA, a novel algorithm that treats protein sequences as a language where mutations create evolutionary families of units rather than isolated tokens. By organizing these variants into genealogical structures, the method improves molecular function representations and accelerates training for protein language models. The approach offers a biologically grounded vocabulary that enhances downstream tasks while reducing embedding table sizes.
PUMA: Learning a Mutation-Aware Vocabulary of Protein Units
arXiv:2503.08838v3 Announce Type: replace Abstract: Modeling protein sequences as a language has made language models a powerful tool in computational biology, yet the language itself remains poorly understood. A key step toward understanding it is identifying its constituent units. In natural languages, morphemes can occur in multiple forms; similarly, in proteins, mutations can give rise to variations of a unit that persist through evolution, forming families of related units. We introduce PUMA (Protein Units via Mutation-Aware Merging), an algorithm that learns protein units from sequence and explores their mutational variants using substitution matrices, forming a genealogy of unit families. Our results show that mutations remaining within a PUMA family are more often benign than the substitution matrix alone predicts, and that PUMA genealogy improves molecular function representations compared to treating units independently. A case study of a unit family demonstrates relatedness beyond homology. PUMA achieves competitive performance on downstream tasks when used as a protein language model tokenizer. Moreover, collapsing units into families results in a smaller embedding table and faster training. Together, these results support PUMA as a biologically grounded protein vocabulary that organizes protein units into plausible families of mutational variants. The source code is available at https://github.com/boun-tabi-lifelu/PUMA.
Abstract:Modeling protein sequences as a language has made language models a powerful tool in computational biology, yet the language itself remains poorly understood. A key step toward understanding it is identifying its constituent units. In natural languages, morphemes can occur in multiple forms; similarly, in proteins, mutations can give rise to variations of a unit that persist through evolution, forming families of related units. We introduce PUMA (Protein Units via Mutation-Aware Merging), an algorithm that learns protein units from sequence and explores their mutational variants using substitution matrices, forming a genealogy of unit families. Our results show that mutations remaining within a PUMA family are more often benign than the substitution matrix alone predicts, and that PUMA genealogy improves molecular function representations compared to treating units independently. A case study of a unit family demonstrates relatedness beyond homology. PUMA achieves competitive performance on downstream tasks when used as a protein language model tokenizer. Moreover, collapsing units into families results in a smaller embedding table and faster training. Together, these results support PUMA as a biologically grounded protein vocabulary that organizes protein units into plausible families of mutational variants. The source code is available at this https URL.
Computation and Language (cs.CL); Quantitative Methods (q-bio.QM)
arXiv:2503.08838 [cs.CL]
(or arXiv:2503.08838v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2503.08838 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 293 words
Related coverage
News
Quantum Computing Report · Jul 25, 2026
Constraint-Aware Optimization for Robust Protein Stability Prediction
arXiv · Jun 9, 2026
Structure-aware Reinforcement Learning for Protein Directed Evolution
arXiv · Oct 1, 2026
Beyond Rule-Based Mutation Testing: Test-Aware Mutant Generation Using Large Language Models
arXiv · Sep 30, 2026