analiza aiqbench
GIFT addresses the challenge of non-stationary financial markets by using LLMs solely to refine state-reward interfaces for PPO agents, rather than making direct trading decisions. The framework enhances learning signals through factor-guided state features and risk-rule-based rewards, ultimately improving out-of-sample risk-adjusted performance without requiring LLM queries during evaluation. This approach offers a practical pathway for integrating domain knowledge into reinforcement learning systems while maintaining computational efficiency at test time.
GIFT: LLM-Guided State-Reward Interface for Financial Reinforcement Learning
arXiv:2606.08450v1 Announce Type: new Abstract: Financial portfolio trading is naturally formulated as a reinforcement learning problem, where an agent sequentially rebalances assets under changing market conditions to balance return, risk, and transaction costs. Yet in non-stationary markets, raw OHLCV states and short-horizon return rewards often provide an under-specified learning interface, motivating large language models as a way to inject financial knowledge into state and reward design while constraining open-ended generation. To this end, we propose GIFT, an LLM-guided framework for state-reward interface design in PPO-based financial reinforcement learning. Rather than using the LLM to make trading decisions, GIFT uses Factor-guided State Enhancement to generate state features from financial-factor primitives, Risk-rule-guided Reward Shaping to generate auxiliary rewards from portfolio-risk rules, and Diagnostic-guided Refinement to revise candidate interfaces using PPO rollout diagnostics. After refinement, GIFT fixes the selected state-reward interface before evaluation, with no further LLM queries or interface updates at test time. Comprehensive rolling-window experiments across diverse market regimes and portfolio scenarios demonstrate that GIFT improves learning-signal quality and out-of-sample risk-adjusted portfolio performance over baselines. Code and data are available at: https://github.com/KAG778/GIFT .
Abstract:Financial portfolio trading is naturally formulated as a reinforcement learning problem, where an agent sequentially rebalances assets under changing market conditions to balance return, risk, and transaction costs. Yet in non-stationary markets, raw OHLCV states and short-horizon return rewards often provide an under-specified learning interface, motivating large language models as a way to inject financial knowledge into state and reward design while constraining open-ended generation. To this end, we propose GIFT, an LLM-guided framework for state-reward interface design in PPO-based financial reinforcement learning. Rather than using the LLM to make trading decisions, GIFT uses Factor-guided State Enhancement to generate state features from financial-factor primitives, Risk-rule-guided Reward Shaping to generate auxiliary rewards from portfolio-risk rules, and Diagnostic-guided Refinement to revise candidate interfaces using PPO rollout diagnostics. After refinement, GIFT fixes the selected state-reward interface before evaluation, with no further LLM queries or interface updates at test time. Comprehensive rolling-window experiments across diverse market regimes and portfolio scenarios demonstrate that GIFT improves learning-signal quality and out-of-sample risk-adjusted portfolio performance over baselines. Code and data are available at: this https URL .
25 pages, 7 figures. Code and data are available at this https URL . Equal contribution: Yanyan Wu and Boyi Zhang. Corresponding author: Youhua Li
Artificial Intelligence (cs.AI)
arXiv:2606.08450 [cs.AI]
(or arXiv:2606.08450v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2606.08450 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 222 cuvinte