analiza aiqbench
This research introduces a differentiable, GPU-accelerated acoustic simulator that solves the inverse problem of reconstructing vocal tract geometry solely from audio signals. By leveraging gradient descent and neural field parameterizations, the method enables seamless integration with deep learning pipelines for applications in linguistics and medical imaging without requiring paired MRI data. The work represents a significant advancement in physics-informed machine learning for speech synthesis and analysis.
Reconstructing the Vocal Tract with Differentiable Acoustic Simulation
arXiv:2609.36737v1 Announce Type: cross Abstract: The vocal tract is the region of the human body responsible for filtering one's voice to create speech. In this paper, we present a differentiable and GPU accelerated acoustic simulator for the vocal tract. The differentiable simulator synthesizes speech by propagating sound along an acoustic tube model of the vocal tract, and via its gradients, can solve the inverse problem: reconstructing the shape of the vocal tract solely from the sound it produces. Although the inverse mapping between geometry and sound is notoriously non-convex, we discover that gradient descent succeeds with three technical contributions: (1) we design a frequency domain formulation of the vocal tract's fluid dynamics that is 70x more GPU parallelizable than finite differences in time, (2) we integrate a differentiable model for turbulence to synthesize consonants, and (3) similar to prior work in implicit neural representations (INRs) and neural fields, we find that parameterizing the geometry with a neural network accelerates convergence and escapes local minima that trap discrete representations. Because the simulator is differentiable, it is readily integrated with other deep learning pipelines to enable novel linguistics and medical imaging applications. (1) We demonstrate self-supervised autoencoding of vocal tract shapes across 11 languages, and (2) we couple our simulator with a generative model of MRI (magnetic resonance imaging) images to reconstruct one's moving vocal tract from only their speech without paired data.
Abstract:The vocal tract is the region of the human body responsible for filtering one's voice to create speech. In this paper, we present a differentiable and GPU accelerated acoustic simulator for the vocal tract. The differentiable simulator synthesizes speech by propagating sound along an acoustic tube model of the vocal tract, and via its gradients, can solve the inverse problem: reconstructing the shape of the vocal tract solely from the sound it produces. Although the inverse mapping between geometry and sound is notoriously non-convex, we discover that gradient descent succeeds with three technical contributions: (1) we design a frequency domain formulation of the vocal tract's fluid dynamics that is 70x more GPU parallelizable than finite differences in time, (2) we integrate a differentiable model for turbulence to synthesize consonants, and (3) similar to prior work in implicit neural representations (INRs) and neural fields, we find that parameterizing the geometry with a neural network accelerates convergence and escapes local minima that trap discrete representations. Because the simulator is differentiable, it is readily integrated with other deep learning pipelines to enable novel linguistics and medical imaging applications. (1) We demonstrate self-supervised autoencoding of vocal tract shapes across 11 languages, and (2) we couple our simulator with a generative model of MRI (magnetic resonance imaging) images to reconstruct one's moving vocal tract from only their speech without paired data.
Accepted as NeurIPS 2026 spotlight paper. Supplementary material at this https URL
Sound (cs.SD); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS)
arXiv:2609.36737 [cs.SD]
(or arXiv:2609.36737v1 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2609.36737 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 340 cuvinte
Subiecte conexe
Structure-Preserving Quantum Simulation of Wave Equations on a Trapped-Ion Processor
arXiv · 31 iul. 2026
Robust Ensemble Guidance for Scientific Inverse Problems
arXiv · 6 oct. 2026
Accelerating physics-informed neural networks for full waveform inversion using a hybrid quantum-classical finite-basis architecture
arXiv · 16 iun. 2026
Physics-Informed Neural Engine Sound Modeling with Differentiable Pulse-Train Synthesis
arXiv · 4 iun. 2026