analiza aiqbench
This research introduces NeST, a framework that enhances Large Language Models for time series forecasting by aligning textual prototypes with temporal data features. The approach achieves measurable improvements in accuracy across multiple benchmarks and real-world energy grid applications without relying on heavy cross-attention mechanisms.
NeST: Neighborhood-aware semantic alignment and temporal modulation for LLM based time series forecasting
arXiv:2412.04806v2 Announce Type: replace-cross Abstract: Adapting Large Language Models (LLMs) trained on discrete text data, to forecast continuous time series signals is challenging. While finetuning the LLMs enables such adaptation, effectively integrating both textual and time series information in the prompt is critical. Current LLM-based time series forecasting methods combine the two modalities through simple concatenation or parameter heavy cross-attention. Moreover, existing methods embed time series data using decomposition techniques that may inadequately capture complex temporal dynamics. To address these limitations, we propose neighborhood-aware semantic alignment and temporal modulation based framework (NEST) to formulate a new text-integrated time series prompt to finetune the LLM. First, we generate neighborhood-aware text prototypes that are optimized to represent local neighborhoods of pretrained word token embeddings of the LLM. Second, we align them with temporal representations of the time series input using a nearest-neighbor contrastive objective, after which the top-k most relevant text prototypes are retrieved. Third, we introduce text prototype conditioned temporal modulation that uses the retrieved text prototypes to adaptively scale and shift time series features. Extensive experiments demonstrate that NeST consistently outperforms state-of-the-art methods across eight benchmarks, achieving an average 1.2\% reduction in MSE for long-term forecasting. In addition, it demonstrates strong generalization, yielding an average 4.9\% reduction in MSE in zero-shot forecasting. Beyond benchmark datasets, NeST also delivers robust performance on a real-world distributed photovoltaic power forecasting task across nine datasets, improving the average R$^2$ score by 3.3\%. These findings demonstrate the effectiveness and generalizability of NeST for adapting LLMs to diverse time series forecasting tasks.
Abstract:Adapting Large Language Models (LLMs) trained on discrete text data, to forecast continuous time series signals is challenging. While finetuning the LLMs enables such adaptation, effectively integrating both textual and time series information in the prompt is critical. Current LLM-based time series forecasting methods combine the two modalities through simple concatenation or parameter heavy cross-attention. Moreover, existing methods embed time series data using decomposition techniques that may inadequately capture complex temporal dynamics. To address these limitations, we propose neighborhood-aware semantic alignment and temporal modulation based framework (NEST) to formulate a new text-integrated time series prompt to finetune the LLM. First, we generate neighborhood-aware text prototypes that are optimized to represent local neighborhoods of pretrained word token embeddings of the LLM. Second, we align them with temporal representations of the time series input using a nearest-neighbor contrastive objective, after which the top-k most relevant text prototypes are retrieved. Third, we introduce text prototype conditioned temporal modulation that uses the retrieved text prototypes to adaptively scale and shift time series features. Extensive experiments demonstrate that NeST consistently outperforms state-of-the-art methods across eight benchmarks, achieving an average 1.2\% reduction in MSE for long-term forecasting. In addition, it demonstrates strong generalization, yielding an average 4.9\% reduction in MSE in zero-shot forecasting. Beyond benchmark datasets, NeST also delivers robust performance on a real-world distributed photovoltaic power forecasting task across nine datasets, improving the average R$^2$ score by 3.3\%. These findings demonstrate the effectiveness and generalizability of NeST for adapting LLMs to diverse time series forecasting tasks.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2412.04806 [cs.LG]
(or arXiv:2412.04806v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2412.04806 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 350 cuvinte
Subiecte conexe
Game Design Creativity evals
LessWrong · 12 oct. 2026
STELLAR: Spatio-Temporal Environmental Learning with Latent Alignment and Refinement for Long-Tailed Species Distribution Modeling
arXiv · 9 iun. 2026
Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos
arXiv · 20 iul. 2026
InA-Probe: Instruction-Aware Active Probing for Time Series Forecasting with LLMs
arXiv · 9 iun. 2026