analiza aiqbench
This research addresses the critical challenge of LLM knowledge staleness by introducing a simulation-driven framework that generates synthetic future events to train models without data contamination. The proposed Synapse pipeline demonstrates that using model-generated trajectories for mid-training and instruction tuning significantly improves the robustness and coherence of knowledge insertion compared to existing methods. By decoupling evaluation from static corpora, this work offers a scalable path toward maintaining up-to-date AI systems without relying on expensive human-curated datasets.
Synthetic Worlds for Temporal Evaluation and Knowledge Updating in LLMs
arXiv:2609.00184v1 Announce Type: cross Abstract: Large language models (LLMs) rely on static pretraining corpora, causing their knowledge to become outdated over time. Existing approaches for evaluating knowledge edits either suffer from rapid contamination or rely on counterfactual edits that conflict with rigid existing knowledge. In this work, we propose a synthetic, simulation-driven framework for studying knowledge insertion in LLMs. We introduce {\sc ParallelEvents}, a benchmark of fictional yet realistic future worlds that generates coherent event trajectories for controlled evaluation, avoiding contamination while preserving consistency. Building on this dataset, we develop {\sc Synapse}, a training framework that uses model-generated data to update model parameters via mid-training and instruction tuning. This synthetic pipeline enables scalable knowledge integration without costly human-curated data. Empirically, {\sc Synapse} outperforms existing methods by 14.23\%, demonstrating that simulation-based synthetic training leads to robust and coherent knowledge insertions.
Abstract:Large language models (LLMs) rely on static pretraining corpora, causing their knowledge to become outdated over time. Existing approaches for evaluating knowledge edits either suffer from rapid contamination or rely on counterfactual edits that conflict with rigid existing knowledge. In this work, we propose a synthetic, simulation-driven framework for studying knowledge insertion in LLMs. We introduce {\sc ParallelEvents}, a benchmark of fictional yet realistic future worlds that generates coherent event trajectories for controlled evaluation, avoiding contamination while preserving consistency. Building on this dataset, we develop {\sc Synapse}, a training framework that uses model-generated data to update model parameters via mid-training and instruction tuning. This synthetic pipeline enables scalable knowledge integration without costly human-curated data. Empirically, {\sc Synapse} outperforms existing methods by 14.23\%, demonstrating that simulation-based synthetic training leads to robust and coherent knowledge insertions.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2609.00184 [cs.CL]
(or arXiv:2609.00184v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.00184 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 232 cuvinte
Subiecte conexe
What's Continual Learning, and Why Might We Expect To See It In Advanced LLM Agents?
LessWrong · 12 iun. 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · 25 sept. 2026
KC-Agent: A Dual-Process Cognitive Architecture for Efficient ML Model Improvement
arXiv · 5 aug. 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · 29 sept. 2026