aiqbench analysis
This arXiv paper proposes a common framework for comparing LLM-based "Experience Managers" in interactive stories, using a shared symbolic world model and state-transition plans to score player agency, controllability, author intent, and robustness regardless of how each manager works internally. Its evaluation of three managers on a single Little Red Riding Hood scenario is small-scale, so the findings are preliminary. The useful takeaway is that no manager dominated on every property, so the right choice depends on whether an author prioritises freedom or control.
Creating Hybrid Interactive Narrative Systems with LLM: From Formal Description to Evaluation
arXiv:2611.02807v1 Announce Type: new Abstract: Interactive narratives let users shape a story while an Experience Manager (EM) keeps it coherent, balancing player freedom against authorial control. Large Language Models now enable a new generation of hybrid EMs, which combine run-time generation from free-text input with symbolic narrative control. However, their heterogeneity makes them hard to compare: A manager's behaviour is generally tied to the unique and specific scenario in which it operates, and much of its decision-making takes place outside of any explicit and verifiable structure. We address this with an LLM-based narrative platform whose managers are interchangeable modules operating over a shared symbolic world model of facts, operations, and goals. Because every applied operation carries explicit preconditions and effects, each session can be analysed as a plan through a common state-transition representation and evaluated, independently of how the manager works internally, with shared indicators of four properties: player agency, EM controllability, author intent, and robustness. We implement three contrasting managers, based respectively on open generation, Monte Carlo repair, and autonomous characters, and compare them through functional analysis on an interactive Little Red Riding Hood scenario. The results show that no manager dominates on every property, giving authors and EM designers a concrete basis to select a manager according to their priorities.
Abstract:Interactive narratives let users shape a story while an Experience Manager (EM) keeps it coherent, balancing player freedom against authorial control. Large Language Models now enable a new generation of hybrid EMs, which combine run-time generation from free-text input with symbolic narrative control. However, their heterogeneity makes them hard to compare: A manager's behaviour is generally tied to the unique and specific scenario in which it operates, and much of its decision-making takes place outside of any explicit and verifiable structure. We address this with an LLM-based narrative platform whose managers are interchangeable modules operating over a shared symbolic world model of facts, operations, and goals. Because every applied operation carries explicit preconditions and effects, each session can be analysed as a plan through a common state-transition representation and evaluated, independently of how the manager works internally, with shared indicators of four properties: player agency, EM controllability, author intent, and robustness. We implement three contrasting managers, based respectively on open generation, Monte Carlo repair, and autonomous characters, and compare them through functional analysis on an interactive Little Red Riding Hood scenario. The results show that no manager dominates on every property, giving authors and EM designers a concrete basis to select a manager according to their priorities.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)
arXiv:2611.02807 [cs.AI]
(or arXiv:2611.02807v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2611.02807 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 308 words
Related coverage
When Seeing Is Not Enough: Benchmarking Interactive Visual Grounding in LVLMs
arXiv · Aug 27, 2026
Non-Factorisable Systems Always Exhibit Deviation from Linear Quantum Dynamics
Quantum Zeitgeist · Aug 22, 2026
Human--AI Interactions as First-Class Data: A Formal Interaction Model and Runtime for Explainable AI Collaboration
arXiv · Oct 31, 2026
SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environments
arXiv · Sep 3, 2026