aiqbench analysis
MolGram addresses the locality gap in molecular language models by integrating a conditional $n$-gram memory module that efficiently captures chemically meaningful motifs without altering standard tokenizers. This approach demonstrates significant performance gains across generation and retrosynthesis tasks while requiring fewer parameters than high-capacity baselines, validating explicit local pattern memory as a strong inductive bias.
Augmenting Molecular Language Models with Local $n$-gram Memory
arXiv:2606.12113v1 Announce Type: new Abstract: Transformer-based language models for SMILES strings suffer from a locality gap: standard character-level tokenization fragments chemically meaningful motifs, forcing models to repeatedly learn local syntax at the expense of long-range dependencies. To address this without disrupting standard tokenizers, we propose MolGram, which integrates a conditional $n$-gram memory module into molecular language models. MolGram maps local string patterns to learned embeddings via scalable hash lookups and dynamically injects this regional context into hidden states. Evaluations across three tasks, including unconditional molecule generation, forward reaction prediction, and single-step retrosynthesis, show that MolGram consistently improves performance. Crucially, our analyses demonstrate that MolGram outperforms baselines with 3$\times$ more parameters, establishing explicit local pattern memory as a highly efficient inductive bias.
Abstract:Transformer-based language models for SMILES strings suffer from a locality gap: standard character-level tokenization fragments chemically meaningful motifs, forcing models to repeatedly learn local syntax at the expense of long-range dependencies. To address this without disrupting standard tokenizers, we propose MolGram, which integrates a conditional $n$-gram memory module into molecular language models. MolGram maps local string patterns to learned embeddings via scalable hash lookups and dynamically injects this regional context into hidden states. Evaluations across three tasks, including unconditional molecule generation, forward reaction prediction, and single-step retrosynthesis, show that MolGram consistently improves performance. Crucially, our analyses demonstrate that MolGram outperforms baselines with 3$\times$ more parameters, establishing explicit local pattern memory as a highly efficient inductive bias.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2606.12113 [cs.CL]
(or arXiv:2606.12113v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.12113 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 214 words
Related coverage
MolE-RAG: Molecular Structure-Enhanced Retrieval-Augmented Generation for Chemistry
arXiv · Jun 16, 2026
Molecular Property Prediction under Structural Shift with Tabular Foundation Models
arXiv · Oct 1, 2026
Post-Optimization Subspace Corrections to Quantum Ans\"atze
arXiv · Oct 15, 2026
MolDesignBench: Evaluating LLM-based Agent for Scenario-grounded Molecular Design
arXiv · Sep 24, 2026