analiza aiqbench
OTIS challenges the prevailing scaling law hypothesis in time series analysis by demonstrating that a compact 7.1M parameter encoder can outperform models 54 times larger across 162 tasks while drastically reducing memory, energy, and latency requirements. By integrating domain-aware tokenization with dual masking strategies tailored to spatiotemporal structures, this approach effectively removes the capacity tax typically associated with high performance on resource-constrained devices like wearables and industrial sensors. The release of open weights and code further democratizes access to efficient feature extraction for real-world deployment without relying on massive infrastructure.
OTIS: Learning High-Quality Time Series Features With Tiny Encoders
arXiv:2410.07299v3 Announce Type: replace Abstract: We introduce OTIS, an open time series encoder that yields high-quality time series features for downstream deployment on any system, including resource-constrained wearables and industrial sensors. Currently, the development of powerful general-purpose encoders relies on the scaling laws hypothesis, using large encoder sizes to memorise the heterogeneous distributions of multi-domain training data. However, this reliance on scale creates a barrier to real-world utility, rendering deployment on resource-constrained systems infeasible due to strict memory, energy, and latency constraints. Surprisingly, we find that tailoring standard masked modelling pre-training to time series properties yields a tiny $7.1\,$M encoder that matches the state-of-the-art performance of $54\times$ larger encoders across $162$ tasks, while requiring $10\times$ less memory, $43\times$ less energy, and $37\times$ lower latency. To achieve this without the capacity tax, we introduce three novel components: (1) a domain-aware tokeniser to resolve conflicting semantics within multi-domain training data; (2) a dual masking strategy to capture spatiotemporal structures and temporal causality; and (3) a structure-aware objective to decouple feature learning from modelling noise. Consequently, OTIS produces high-quality time series features that enable state-of-the art performance in discriminative tasks and even extend seamlessly to generative tasks at minimal additional cost. To democratise access to powerful time series features on any system, we release our code and pre-trained weights.
Abstract:We introduce OTIS, an open time series encoder that yields high-quality time series features for downstream deployment on any system, including resource-constrained wearables and industrial sensors. Currently, the development of powerful general-purpose encoders relies on the scaling laws hypothesis, using large encoder sizes to memorise the heterogeneous distributions of multi-domain training data. However, this reliance on scale creates a barrier to real-world utility, rendering deployment on resource-constrained systems infeasible due to strict memory, energy, and latency constraints. Surprisingly, we find that tailoring standard masked modelling pre-training to time series properties yields a tiny $7.1\,$M encoder that matches the state-of-the-art performance of $54\times$ larger encoders across $162$ tasks, while requiring $10\times$ less memory, $43\times$ less energy, and $37\times$ lower latency. To achieve this without the capacity tax, we introduce three novel components: (1) a domain-aware tokeniser to resolve conflicting semantics within multi-domain training data; (2) a dual masking strategy to capture spatiotemporal structures and temporal causality; and (3) a structure-aware objective to decouple feature learning from modelling noise. Consequently, OTIS produces high-quality time series features that enable state-of-the art performance in discriminative tasks and even extend seamlessly to generative tasks at minimal additional cost. To democratise access to powerful time series features on any system, we release our code and pre-trained weights.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2410.07299 [cs.LG]
(or arXiv:2410.07299v3 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2410.07299 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 312 cuvinte
Subiecte conexe
Vertical Advantage: Transforming Industries with Lakebase and Agentic AI
Databricks · 27 aug. 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · 25 sept. 2026
Why do models task game?
AI Alignment Forum · 6 aug. 2026
Microsoft’s AI chief says superintelligence is near, but won’t take your job
The Verge · 8 iun. 2026