analiza aiqbench
This paper applies online reinforcement learning to personalize activity suggestions in a digital health trial, with the main methodological contribution being how an expert-built causal DAG shapes state construction, handling of partially observed states, and delayed credit assignment. The authors also use a simulator calibrated from prior trial data to compare RL designs before deployment. For ML readers, the useful takeaway is the practical handling of sparse per-user data, delayed and noisy rewards, and burden-versus-benefit trade-offs. The results are still forthcoming, so the claims remain design-level rather than demonstrated outcomes.
Causal DAG-Guided Reinforcement Learning for Long-Term Physical Activity Maintenance: Using Prior Data to Inform Future Deployment
arXiv:2610.36019v1 Announce Type: cross Abstract: Digital health interventions can provide scalable, personalized support for physical activity (PA), yet sustaining PA over the long-term remains difficult. Because affective experiences with PA are important for activity sustainment, we developed an online reinforcement learning (RL) system for the forthcoming ADAPT factorial trial that personalizes twice-daily activity suggestions aimed at improving affective associations with PA. This setting poses several challenges: participant-specific data are scarce relative to the candidate state space, important states and rewards are partially observed, interventions are frequent while the primary outcome is measured weekly, and activity suggestions can both improve PA-related experiences and increase intervention burden. We address these challenges using an expert-specified causal directed acyclic graph as an integrated RL design tool, guiding state construction, estimation of partially observed states, delayed credit assignment, and representation of beneficial and adverse intervention pathways. We further construct a longitudinal simulator calibrated from a prior micro-randomized trial to compare candidate RL designs under realistic state evolution, missingness, and participant heterogeneity before deployment. Together, this work illustrates how causal domain knowledge and prior trial data can make long-horizon, participant-specific RL more practical for digital health interventions targeting sustained behavior change.
Abstract:Digital health interventions can provide scalable, personalized support for physical activity (PA), yet sustaining PA over the long-term remains difficult. Because affective experiences with PA are important for activity sustainment, we developed an online reinforcement learning (RL) system for the forthcoming ADAPT factorial trial that personalizes twice-daily activity suggestions aimed at improving affective associations with PA. This setting poses several challenges: participant-specific data are scarce relative to the candidate state space, important states and rewards are partially observed, interventions are frequent while the primary outcome is measured weekly, and activity suggestions can both improve PA-related experiences and increase intervention burden. We address these challenges using an expert-specified causal directed acyclic graph as an integrated RL design tool, guiding state construction, estimation of partially observed states, delayed credit assignment, and representation of beneficial and adverse intervention pathways. We further construct a longitudinal simulator calibrated from a prior micro-randomized trial to compare candidate RL designs under realistic state evolution, missingness, and participant heterogeneity before deployment. Together, this work illustrates how causal domain knowledge and prior trial data can make long-horizon, participant-specific RL more practical for digital health interventions targeting sustained behavior change.
Applications (stat.AP); Machine Learning (cs.LG)
arXiv:2610.36019 [stat.AP]
(or arXiv:2610.36019v1 [stat.AP] for this version)
https://doi.org/10.48550/arXiv.2610.36019 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 286 cuvinte
Subiecte conexe
Multilingual Sentiment Aware Text Summarization A Reinforcement Learning Approach for Consistency Maintenance
arXiv · 9 iun. 2026
Emergence, Not Bandwidth: Physical Coupling and the Limits of Learned Multi-Agent Communication
arXiv · 29 sept. 2026
Planning or Learning: Reliability and Cost in Multi-Asset Maintenance
arXiv · 16 sept. 2026
Graph-Based Inference and Topology-Aware Multi-Agent Reinforcement Learning for Large-Scale Railway Network Management
arXiv · 26 sept. 2026