Computational Onboard Data Management for Heterogeneous Autonomous Vehicle Streams

Abstract:Autonomous vehicles continuously generate large volumes of heterogeneous sensor and state data streams. While this data supports real-time autonomy, it also enables emerging vehicle computing applications that require retained historical context. Existing in-vehicle logging stacks are largely replay- or compliance-oriented: rich sensor streams are often discarded after online processing or recorded only in limited windows for offline replay. This creates a new onboard vehicle-data operating point: transforming transient multimodal streams into compact, long-horizon, queryable history under embedded resource constraints. We present Autonomous Vehicle Storage (AVS), a computational and hierarchical in-vehicle data management system for this operating point. AVS introduces a vehicle-specialized architecture including (i) utility-aware modality preprocessing before persistence, (ii) a chunk-indexed append-only logger for sustained ingest, time-range lookup, and prefix recovery, and (iii) an SSD-HDD hot/cold hierarchy that supports direct cold queries. Evaluated on a real L4 platform with three days of traces and concurrent LiDAR, camera, and GPS logging, AVS reduces storage by 8.0-8.7x relative to raw rosbag2 from its modality and task aware preprocessing, delivers hot-tier (SSD) query Time To First Byte (TTFB) of 0.9-9.1 ms and cold-tier (HDD) query TTFB of 25.6-44.0 ms, recovers from power loss in 9.98 s with a bounded 0.24 s loss window, and sustains background archival to HDD at about 107-108 MB/s. It shows the feasibility of AVS on real in-vehicle platforms, and it also exposes key tradeoffs and open challenges for vehicle data engineering.

Extended abstract presented at The 37th Magnetic Recording Conference (TMRC 2026), Paper E-7. 2 pages

Distributed, Parallel, and Cluster Computing (cs.DC); Databases (cs.DB); Operating Systems (cs.OS); Robotics (cs.RO)

D.4.2; E.4; I.2.9

arXiv:2511.19453 [cs.DC]

(or arXiv:2511.19453v2 [cs.DC] for this version)

https://doi.org/10.48550/arXiv.2511.19453 Focus to learn more arXiv-issued DOI via DataCite

Proceedings of the 37th Magnetic Recording Conference (TMRC 2026), p. E-7

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.