aiqbench analysis
This research introduces LC-SEPLM, a method that integrates long-range residue contact supervision into protein language models to enhance their ability to infer 3D structural information from sequences alone. By adapting the ESM2 architecture with LoRA and specific cross-attention mechanisms, the model achieves significant performance gains in remote-homology recognition while maintaining its sequence-only inference capability. The study validates that explicit spatial constraints can be effectively bounded within standard language modeling frameworks without requiring direct 3D input during downstream tasks.
LC-SEPLM: long-range contact-supervised adaptation for sequence-only protein representation learning
arXiv:2607.22777v1 Announce Type: cross Abstract: Protein language models learn transferable sequence representations. However, because they primarily model contextual dependencies along amino-acid sequences, their training objectives do not explicitly constrain the model to learn three-dimensional residue contacts formed after folding . Here, we introduce LC-SEPLM (Long-range Contact-supervised ESM Protein Language Model), which adapts ESM2 with LoRA and long-range residue-pair contact supervision while retaining sequence-only downstream inference. Pair-specific queries use cross-attention over the complete sequence to extract global sequence context associated with long-range spatial contacts. To expose the model to diverse structural information, we trained LC-SEPLM on 500,000 AlphaFold Swiss-Prot proteins. In downstream evaluation, LC-SEPLM improved all eight protein-level tasks relative to ESM2. The largest gain occurred in remote-homology recognition, where macro-F1 increased from 0.6122 to 0.6769 (+0.0647, or 6.47 percentage points). On the official ESM-S EC benchmark, LC-SEPLM also outperformed ESM-S with a maximum absolute gain of 0.1771. These results support residue-pair contact supervision as a bounded route for introducing structural information into protein sequence representations while preserving sequence-only inference.
Abstract:Protein language models learn transferable sequence representations. However, because they primarily model contextual dependencies along amino-acid sequences, their training objectives do not explicitly constrain the model to learn three-dimensional residue contacts formed after folding . Here, we introduce LC-SEPLM (Long-range Contact-supervised ESM Protein Language Model), which adapts ESM2 with LoRA and long-range residue-pair contact supervision while retaining sequence-only downstream inference. Pair-specific queries use cross-attention over the complete sequence to extract global sequence context associated with long-range spatial contacts. To expose the model to diverse structural information, we trained LC-SEPLM on 500,000 AlphaFold Swiss-Prot proteins. In downstream evaluation, LC-SEPLM improved all eight protein-level tasks relative to ESM2. The largest gain occurred in remote-homology recognition, where macro-F1 increased from 0.6122 to 0.6769 (+0.0647, or 6.47 percentage points). On the official ESM-S EC benchmark, LC-SEPLM also outperformed ESM-S with a maximum absolute gain of 0.1771. These results support residue-pair contact supervision as a bounded route for introducing structural information into protein sequence representations while preserving sequence-only inference.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2607.22777 [cs.LG]
(or arXiv:2607.22777v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2607.22777 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 261 words
Related coverage
The Machines that Make the Machines
NVIDIA · Oct 7, 2026
Building a Foundation Stack for General-Purpose Robots
IEEE Spectrum · Jul 13, 2026
EndPrompt: Efficient Long-Context Extension via Terminal Anchoring
arXiv · Sep 9, 2026
EndPrompt: Efficient Long-Context Extension via Terminal Anchoring
arXiv · Jun 3, 2026