aiqbench analysis
SkillSmith addresses a critical gap in agentic systems by enabling large language models to natively reason over model weights alongside textual data. This approach unifies parametric skill libraries and compositional knowledge, demonstrating that integrating these modalities yields performance gains impossible with text-only or weight-space-only methods alone. The research advances the frontier of how LLMs can autonomously synthesize new capabilities through instruction-steered parameter synthesis.
SkillSmith: Learning to Compose Parametric Skills and Textual Knowledge
arXiv:2607.27497v1 Announce Type: new Abstract: Agentic systems driven by large language models (LLMs) regularly feature two key mechanisms to autonomously solve complex problems: synthesizing text-based knowledge and procedures from past experiences and building parametric (weight-space) skill libraries for recurring sub-goals. To date, research has largely treated these as orthogonal pursuits: either organizing textual knowledge through composition and reflection, or consolidating parametric skills via weight-space merging. Consequently, the seamless integration of text and model weights for targeted performance improvements remains largely unexplored. This work bridges this modality gap by treating model weights as an additional modality that an LLM can natively reason over. We instantiate parametric learning via prefix-tuning and augment an LLM to ingest both prefix weights and rich textual data which capture relationships to a target capability. Our augmented LLM, which we call SkillSmith, synthesizes these inputs to perform instruction-steered parametric synthesis, directly outputting new prefix weights that manifest the target skill. We demonstrate that our approach significantly outperforms both text-only and weight-space-only baselines, unlocking performance gains that are out of reach for uni-modal (text-only or weight-only) adaptations.
Abstract:Agentic systems driven by large language models (LLMs) regularly feature two key mechanisms to autonomously solve complex problems: synthesizing text-based knowledge and procedures from past experiences and building parametric (weight-space) skill libraries for recurring sub-goals. To date, research has largely treated these as orthogonal pursuits: either organizing textual knowledge through composition and reflection, or consolidating parametric skills via weight-space merging. Consequently, the seamless integration of text and model weights for targeted performance improvements remains largely unexplored. This work bridges this modality gap by treating model weights as an additional modality that an LLM can natively reason over. We instantiate parametric learning via prefix-tuning and augment an LLM to ingest both prefix weights and rich textual data which capture relationships to a target capability. Our augmented LLM, which we call SkillSmith, synthesizes these inputs to perform instruction-steered parametric synthesis, directly outputting new prefix weights that manifest the target skill. We demonstrate that our approach significantly outperforms both text-only and weight-space-only baselines, unlocking performance gains that are out of reach for uni-modal (text-only or weight-only) adaptations.
Computation and Language (cs.CL)
arXiv:2607.27497 [cs.CL]
(or arXiv:2607.27497v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2607.27497 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 269 words
Related coverage
SCAFFOLD: Self-Improving Web Agents via Recursive Parametric Skill Abstraction
arXiv · Sep 9, 2026
LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
arXiv · Jun 6, 2026
LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
arXiv · Aug 18, 2026
On-Policy Distillation Teaches New Skills but Not New Knowledge
arXiv · Oct 8, 2026