analiza aiqbench
This paper gives theoretical guarantees for transfer learning in multi-armed bandits when the historical data comes from an unknown, adaptive policy, a common real-world case that existing analyses often skip. It proposes KL-UCB-Transfer, which improves regret constants when source and target means are well separated and reduces to standard KL-UCB otherwise. Its main practical value is the one-sided misspecification analysis, which shows when logarithmic regret survives and when the guarantee is lost.
Transfer Learning in Bandits from Unknown Adaptive Policies: Problem-Dependent Lower and Upper Bounds
arXiv:2610.21582v1 Announce Type: new Abstract: Transfer learning in stochastic multi-armed bandits is typically studied assuming that prior data forms a fixed offline dataset or is generated by a known source. In practice, historical observations come from a previous adaptive decision process whose policy is unknown. We study this setting for bounded rewards in $[0,1]$. Before the online game starts, the learner receives, for each arm $k$, $N'_k$ observations drawn i.i.d.\ from an unknown distribution $\nu'_k$, collected by an arbitrary and possibly data-dependent strategy, so the sample sizes $N'_k$ may be random. Source and target means satisfy $|\mu_k-\mu'_k|<L_k$ for known constants $(L_k)_k$. We derive an asymptotic problem-dependent lower bound and propose KL-UCB-Transfer, which combines online and historical data and achieves logarithmic regret with an improved leading constant whenever $\mu'_a<\mu_1-L_a$, and recovers the classical KL-UCB guarantee otherwise. Since bounding the source-target discrepancy is often unrealistic, we relax the assumption to the one-sided condition $\mu_k-\mu'_k<L_k$ and show that logarithmic regret can survive under misspecification, characterizing three regimes ranging from improvement to a degraded logarithmic bound to a loss of guarantee. Simulations illustrate the practical benefits of exploiting process-generated historical data.
Abstract:Transfer learning in stochastic multi-armed bandits is typically studied assuming that prior data forms a fixed offline dataset or is generated by a known source. In practice, historical observations come from a previous adaptive decision process whose policy is unknown. We study this setting for bounded rewards in $[0,1]$. Before the online game starts, the learner receives, for each arm $k$, $N'_k$ observations drawn i.i.d.\ from an unknown distribution $\nu'_k$, collected by an arbitrary and possibly data-dependent strategy, so the sample sizes $N'_k$ may be random. Source and target means satisfy $|\mu_k-\mu'_k|<L_k$ for known constants $(L_k)_k$. We derive an asymptotic problem-dependent lower bound and propose KL-UCB-Transfer, which combines online and historical data and achieves logarithmic regret with an improved leading constant whenever $\mu'_a<\mu_1-L_a$, and recovers the classical KL-UCB guarantee otherwise. Since bounding the source-target discrepancy is often unrealistic, we relax the assumption to the one-sided condition $\mu_k-\mu'_k<L_k$ and show that logarithmic regret can survive under misspecification, characterizing three regimes ranging from improvement to a degraded logarithmic bound to a loss of guarantee. Simulations illustrate the practical benefits of exploiting process-generated historical data.
Machine Learning (cs.LG)
arXiv:2610.21582 [cs.LG]
(or arXiv:2610.21582v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.21582 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 276 cuvinte
Subiecte conexe
Policy Regret for Embedding Model Routing: Contextual Bandits with Low-Rank Experts
arXiv · 28 sept. 2026
Meta-LinEXP3: Online-within-Online Learning for Adversarial Linear Contextual Bandits
arXiv · 10 sept. 2026
Policy Regret for Embedding Model Routing: Contextual Bandits with Low-Rank Experts
arXiv · 16 iun. 2026
The Single-Copy Quantum Bandit Is Classical: An Exact Spectral Collapse
arXiv · 1 oct. 2026