analiza aiqbench
This research proposes using Implicit Neural Representations (INRs) like SIRENs to compress video and audio by storing them as network weights rather than pixel data, achieving a 2.61x compression ratio against traditional codecs. While the method demonstrates feasibility for streaming via WebRTC prototypes, it currently lags behind standard formats like H.264/HEVC in raw reconstruction quality metrics such as PSNR and SSIM. The study highlights significant trade-offs between model size reduction and fidelity saturation at specific bit-widths, offering a novel but niche approach to media compression driven by deep learning architectures.
Data-driven Video Codec with Implicit Neural Representations
arXiv:2607.15298v1 Announce Type: cross Abstract: A conventional codec stores a video as compressed pixel data. We instead store the video, together with its audio track, as the weights of a single sinusoidal representation network (SIREN) that maps space-time coordinates to RGB values and audio amplitudes. The network uses separate audio and video initialization layers, a stack of shared fully connected hidden layers, and three output branches: one for video and two Siamese audio branches whose disagreement is used to estimate and subtract residual noise. The overfitted teacher network is then compressed by response-based knowledge distillation into a smaller student, followed by 16-bit symmetric weight quantization and lossless LZMA2 (xz) encoding. On a 6.08 MiB test video, the quantized student reaches a video PSNR of 28.72 dB with SSIM of 0.75, and an audio PSNR of 24.18 dB with a log spectral distance of 10.69 dB, while the pipeline shrinks the representation from 9.05 MiB to 2.33 MiB, an overall compression ratio of 2.61. A bit-width sweep from 1-bit to 32-bit quantization shows that reconstruction quality saturates at 16 bits. We compare against H.264, HEVC, and MP3, report where the approach falls short of them, and describe a browser-based prototype that trains, transfers, and decodes these models over WebRTC.
Abstract:A conventional codec stores a video as compressed pixel data. We instead store the video, together with its audio track, as the weights of a single sinusoidal representation network (SIREN) that maps space-time coordinates to RGB values and audio amplitudes. The network uses separate audio and video initialization layers, a stack of shared fully connected hidden layers, and three output branches: one for video and two Siamese audio branches whose disagreement is used to estimate and subtract residual noise. The overfitted teacher network is then compressed by response-based knowledge distillation into a smaller student, followed by 16-bit symmetric weight quantization and lossless LZMA2 (xz) encoding. On a 6.08 MiB test video, the quantized student reaches a video PSNR of 28.72 dB with SSIM of 0.75, and an audio PSNR of 24.18 dB with a log spectral distance of 10.69 dB, while the pipeline shrinks the representation from 9.05 MiB to 2.33 MiB, an overall compression ratio of 2.61. A bit-width sweep from 1-bit to 32-bit quantization shows that reconstruction quality saturates at 16 bits. We compare against H.264, HEVC, and MP3, report where the approach falls short of them, and describe a browser-based prototype that trains, transfers, and decodes these models over WebRTC.
Image and Video Processing (eess.IV); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Sound (cs.SD)
arXiv:2607.15298 [eess.IV]
(or arXiv:2607.15298v1 [eess.IV] for this version)
https://doi.org/10.48550/arXiv.2607.15298 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 304 cuvinte
Subiecte conexe
CogGen: Cognitive-Load-Inspired Fully Unsupervised Deep Generative Modeling for Compressively Sampled MRI Reconstruction
arXiv · 17 iun. 2026
[AINews] High Taste Testers Ep. 1
Latent Space · 17 oct. 2026
Direct Topology Tracking in Continuous Implicit Models
arXiv · 14 sept. 2026
Video-Based Prediction of In-Flight Particle Characteristics in Atmospheric Plasma Spraying
arXiv · 8 iun. 2026