aiqbench analysis
This research introduces a novel pipeline for scientific data compression that combines latent-space Residual Vector Quantization with a U-Net for pixel-space error correction and a Guaranteed Autoencoder stage. By explicitly modeling spatially structured residuals, the method achieves superior compression ratios and lower NRMSE on major simulation datasets while strictly adhering to user-defined error bounds. This advancement is critical for preserving data fidelity in high-stakes scientific simulations where traditional lossy methods often fail to meet accuracy requirements.
Neural Residual Modeling for Scientific Data Compression under Guaranteed Error Bounds
arXiv:2609.23185v1 Announce Type: new Abstract: Lossy compression of scientific simulation data increasingly relies on learned, latent-space architectures such as Residual Vector Quantization (RVQ), which iteratively quantize a base representation and its residuals to progressively reduce reconstruction error. While effective, RVQ performs this residual modeling entirely in latent space, leaving the pixel-space error structure of the reconstruction largely unaddressed. In this work, we propose a post-processing pipeline that augments an RVQ-based compressor with a U-Net trained to predict and correct pixel-space residuals between the original volume and its RVQ reconstruction. We show that these residuals are spatially structured rather than driven by local intensity or gradient features, motivating the need for a deep spatial model rather than simple statistical correction. The U-Net-corrected reconstruction is then passed through a Guaranteed Autoencoder (GAE) stage, which projects the remaining residual onto a per-block PCA basis to enforce a user-specified block-wise error bound. To the best of our knowledge, this is the first pipeline to combine latent-space RVQ, explicit pixel-space residual correction via a deep spatial post-processing network, and GAE-based error-bound guarantees within a single framework for scientific data compression. We evaluate our approach on S3D, JHTDB and E3SM datasets, demonstrating consistent improvements in NRMSE, compression ratio] over RVQ-only and standard residual-correction baselines, while maintaining strict error guarantees required for scientific data fidelity.
Abstract:Lossy compression of scientific simulation data increasingly relies on learned, latent-space architectures such as Residual Vector Quantization (RVQ), which iteratively quantize a base representation and its residuals to progressively reduce reconstruction error. While effective, RVQ performs this residual modeling entirely in latent space, leaving the pixel-space error structure of the reconstruction largely unaddressed. In this work, we propose a post-processing pipeline that augments an RVQ-based compressor with a U-Net trained to predict and correct pixel-space residuals between the original volume and its RVQ reconstruction. We show that these residuals are spatially structured rather than driven by local intensity or gradient features, motivating the need for a deep spatial model rather than simple statistical correction. The U-Net-corrected reconstruction is then passed through a Guaranteed Autoencoder (GAE) stage, which projects the remaining residual onto a per-block PCA basis to enforce a user-specified block-wise error bound. To the best of our knowledge, this is the first pipeline to combine latent-space RVQ, explicit pixel-space residual correction via a deep spatial post-processing network, and GAE-based error-bound guarantees within a single framework for scientific data compression. We evaluate our approach on S3D, JHTDB and E3SM datasets, demonstrating consistent improvements in NRMSE, compression ratio] over RVQ-only and standard residual-correction baselines, while maintaining strict error guarantees required for scientific data fidelity.
Machine Learning (cs.LG)
arXiv:2609.23185 [cs.LG]
(or arXiv:2609.23185v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.23185 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 307 words
Related coverage
Residual Modeling for High-Fidelity Learned Compression of Scientific Data
arXiv · Jun 6, 2026
Can Deep Neural Networks Improve Compression of Very Large Scientific Data?
arXiv · Jun 15, 2026
Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers
Hugging Face · Aug 26, 2026
Introducing PIRAMID: Physics-Informed Research for Ambitious Mechanistic Interpretability
LessWrong · Jul 25, 2026