analiza aiqbench
This research introduces a neurosymbolic framework that integrates Linear Temporal Logic constraints into transformer-based reinforcement learning policies to enforce high-level temporal requirements. By compiling logic formulas into differentiable finite automata, the method achieves improved constraint satisfaction while maintaining competitive returns across various architectures. The approach represents a significant step toward ensuring safety and reachability in offline RL without sacrificing performance.
Neuro-Symbolic Injection of LTLf Constraints in Autoregressive Reinforcement Learning Policies
arXiv:2606.08312v1 Announce Type: new Abstract: In this work we study offline reinforcement learning (RL) under temporally extended task constraints expressed in Linear Temporal Logic over finite traces (LTLf). Recently, transformer-based approaches such as Trajectory Transformers and Decision Transformers have been adopted to address RL as a sequence modeling problem. However, these methods optimize purely for reward and do not account for high-level temporal requirements. Here, we introduce a neurosymbolic framework that injects LTLf background knowledge into such transformer-based RL policies. Our approach compiles LTLf formulas into deterministic finite automata (DFAs) and integrates them into the learning process through a differentiable representation and a logic-based loss function. In particular, we derive differentiable satisfaction signals from DFA progression and use them as a regularization term during training. The resulting method is architecture-agnostic across different models. We evaluate the proposed framework on navigation environments with specification suites covering combinations of safety and reachability temporal properties. Experimental results show that incorporating background knowledge not only improves constraint satisfaction, but also maintains competitive return compared to vanilla baselines.
Abstract:In this work we study offline reinforcement learning (RL) under temporally extended task constraints expressed in Linear Temporal Logic over finite traces (LTLf). Recently, transformer-based approaches such as Trajectory Transformers and Decision Transformers have been adopted to address RL as a sequence modeling problem. However, these methods optimize purely for reward and do not account for high-level temporal requirements. Here, we introduce a neurosymbolic framework that injects LTLf background knowledge into such transformer-based RL policies. Our approach compiles LTLf formulas into deterministic finite automata (DFAs) and integrates them into the learning process through a differentiable representation and a logic-based loss function. In particular, we derive differentiable satisfaction signals from DFA progression and use them as a regularization term during training. The resulting method is architecture-agnostic across different models. We evaluate the proposed framework on navigation environments with specification suites covering combinations of safety and reachability temporal properties. Experimental results show that incorporating background knowledge not only improves constraint satisfaction, but also maintains competitive return compared to vanilla baselines.
Artificial Intelligence (cs.AI); Formal Languages and Automata Theory (cs.FL)
arXiv:2606.08312 [cs.AI]
(or arXiv:2606.08312v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.08312 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 192 cuvinte
Subiecte conexe
Managing Action Preconditions in Neuro-Symbolic RL: Three Placement Strategies for Embodied Agents
arXiv · 16 sept. 2026
Constraint-Anchored Reasoning Traces
arXiv · 21 iul. 2026
Neural Proposals, Symbolic Guarantees: Neuro-Symbolic Graph Generative Modeling
arXiv · 30 sept. 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · 29 sept. 2026