analiza aiqbench
This paper advances decision-focused learning by introducing an on-policy method for contextual linear optimization under partial feedback, addressing limitations in existing offline or full-observation approaches. The authors propose a hybrid gradient estimator that combines score function estimation with a decision-focused plug-in component to improve policy updates while proving convergence rates matching standard non-convex SGD. Experimental results across combinatorial pricing and energy scheduling benchmarks demonstrate superior regret performance compared to contextual-bandit baselines, offering practical improvements for sequential optimization tasks with limited data visibility.
Decision-Focused On-Policy Learning for Contextual Linear Optimization with Partial Feedback
arXiv:2606.01081v2 Announce Type: replace Abstract: Decision-focused learning (DFL) trains predictive models by optimizing downstream decision quality rather than standalone prediction accuracy. For contextual linear optimization, most existing DFL methods assume offline data and full observations of the objective cost vector. We develop an on-policy learning method for sequential contextual linear optimization under partial feedback, generalizing the standard bandit feedback setting. Our method learns a stochastic predict-then-optimize policy that samples a cost-vector prediction from a conditional distribution and solves the resulting downstream linear optimization problem. To update this distributional model, we introduce a two-component hybrid gradient estimator. The first component is a score function estimator, which provides an unbiased but potentially high-variance policy gradient estimate. The second is a decision-focused plug-in component that uses an auxiliary nuisance estimate of the latent cost vector to exploit the downstream optimization structure, becoming more informative as the estimate improves. We prove an O(T^-1/2) bound on the average squared policy-gradient norm, matching the standard non-convex SGD rate. Experiments on top-k selection, shortest path, combinatorial pricing, and a real-data energy-scheduling benchmark show that, under bandit feedback, the hybrid gradient approach achieves lower mean cumulative regret than the contextual-bandit baselines on all four benchmarks and than linear Thompson sampling on three, and that it also works with richer conditional generative cost models. Code is available at https://github.com/Joeyetinghan/on-policy-bandit-dfl.
Abstract:Decision-focused learning (DFL) trains predictive models by optimizing downstream decision quality rather than standalone prediction accuracy. For contextual linear optimization, most existing DFL methods assume offline data and full observations of the objective cost vector. We develop an on-policy learning method for sequential contextual linear optimization under partial feedback, generalizing the standard bandit feedback setting. Our method learns a stochastic predict-then-optimize policy that samples a cost-vector prediction from a conditional distribution and solves the resulting downstream linear optimization problem. To update this distributional model, we introduce a two-component hybrid gradient estimator. The first component is a score function estimator, which provides an unbiased but potentially high-variance policy gradient estimate. The second is a decision-focused plug-in component that uses an auxiliary nuisance estimate of the latent cost vector to exploit the downstream optimization structure, becoming more informative as the estimate improves. We prove an O(T^-1/2) bound on the average squared policy-gradient norm, matching the standard non-convex SGD rate. Experiments on top-k selection, shortest path, combinatorial pricing, and a real-data energy-scheduling benchmark show that, under bandit feedback, the hybrid gradient approach achieves lower mean cumulative regret than the contextual-bandit baselines on all four benchmarks and than linear Thompson sampling on three, and that it also works with richer conditional generative cost models. Code is available at this https URL.
Machine Learning (cs.LG)
arXiv:2606.01081 [cs.LG]
(or arXiv:2606.01081v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.01081 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 309 cuvinte
Subiecte conexe
Vertical Advantage: Transforming Industries with Lakebase and Agentic AI
Databricks · 27 aug. 2026
Deep-learning Causal Retrieval Optimization for Efficient e-commerce Distribution in Pinterest
arXiv · 22 iul. 2026
The Entangled Dimensions of Decision Theory
LessWrong · 30 iul. 2026
Task Specialization Fine-Tuning for Contextual Reinforcement Learning
arXiv · 20 aug. 2026