analiza aiqbench
SAGE introduces a novel CLIP-based framework that injects variable-specific semantic and visual knowledge into time series forecasting without relying on computationally expensive large language models at inference time. By jointly modeling temporal dynamics with grounded encodings, the method achieves state-of-the-art accuracy across multiple benchmarks while addressing the limitations of uniform textual prompts. This approach represents a significant step toward integrating domain expert intuition directly into predictive models through efficient multimodal alignment.
SAGE: Variate-Wise Semantic Augmentation for Vision-Language Time Series Forecasting
arXiv:2608.26829v1 Announce Type: new Abstract: Time series forecasting models operate on raw numerical sequences, lacking the semantic knowledge that domain experts implicitly leverage, such as the physical meaning of each variable, its statistical behavior, and its temporal dynamics. Recent efforts to bridge this gap fall into two camps. Some rely on large language models at inference time, which is computationally expensive. Others apply uniform textual prompts at the dataset level, ignoring the heterogeneous semantics across individual variates. We propose SAGE (Seeing and Augmenting with Grounded Encoding), an end-to-end CLIP-based framework that jointly models temporal, cross-variable, textual, and visual information. The CLIP text encoder processes frequency-enhanced patches and variable tokens, while gated residual paths inject variable-specific descriptions and statistical descriptors. In parallel, the frozen CLIP vision encoder aligns rendered series with temporal representations through a training-only contrastive objective. This dual use of CLIP adds complementary semantic and visual supervision without placing an LLM in the forecasting loop. Across eight long-term benchmarks and M4, SAGE achieves state-of-the-art accuracy. Ablations confirm complementary gains from multimodal alignment and variable-level knowledge.
Abstract:Time series forecasting models operate on raw numerical sequences, lacking the semantic knowledge that domain experts implicitly leverage, such as the physical meaning of each variable, its statistical behavior, and its temporal dynamics. Recent efforts to bridge this gap fall into two camps. Some rely on large language models at inference time, which is computationally expensive. Others apply uniform textual prompts at the dataset level, ignoring the heterogeneous semantics across individual variates. We propose SAGE (Seeing and Augmenting with Grounded Encoding), an end-to-end CLIP-based framework that jointly models temporal, cross-variable, textual, and visual information. The CLIP text encoder processes frequency-enhanced patches and variable tokens, while gated residual paths inject variable-specific descriptions and statistical descriptors. In parallel, the frozen CLIP vision encoder aligns rendered series with temporal representations through a training-only contrastive objective. This dual use of CLIP adds complementary semantic and visual supervision without placing an LLM in the forecasting loop. Across eight long-term benchmarks and M4, SAGE achieves state-of-the-art accuracy. Ablations confirm complementary gains from multimodal alignment and variable-level knowledge.
Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2608.26829 [cs.LG]
(or arXiv:2608.26829v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2608.26829 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 271 cuvinte
Subiecte conexe
When Does Language Matter? Multilingual Instructions Reveal Step-wise Language Sensitivity in Vision-Language-Action Models
arXiv · 11 iun. 2026
FM-VLA: Force-based Memory for Vision-Language-Action Models in Contact-Rich Manipulation
arXiv · 21 iul. 2026
Model Weight Profiles: Where Do the Parameters Go?
AMD · 25 sept. 2026
SPI: Query-Depth-Adaptive Indexing for Streaming RAG in Vector Databases
arXiv · 17 iun. 2026