aiqbench analysis
Synth-JEPA advances sound synthesis by training a joint embedding space that enables direct parameter optimization without the computational cost of real-time audio rendering. This approach significantly outperforms existing inverse models and generic audio objectives, demonstrating that geometry learned from parameter correspondences is superior to standard audio similarity metrics for synthesizer matching. The method effectively trades compute during inference for higher fidelity matches, validated by human listeners in 85% of trials.
Synth-JEPA: Joint Embedding Prediction for Renderer-Free Synthesizer Parameter Search
arXiv:2609.31024v1 Announce Type: cross Abstract: Sound matching can be formulated as optimizing synthesizer parameters against an audio-domain objective. However, objectives derived from generic audio representations are often difficult to optimize, while direct search requires rendering every candidate. We introduce Synth-JEPA, which learns mutually predictive audio and parameter representations from paired synthesizer data. At inference, candidate parameters are scored directly in this learned space, yielding a renderer-free objective whose audio geometry is shaped by parameter correspondences rather than generic audio similarity. We evaluate Synth-JEPA on Surge XT using held-out synthesizer sounds and out-of-domain NSynth and FSD50K targets, against inverse models, direct search, and learned proxy objectives. Synth-JEPA outperforms all baselines in-domain and remains competitive out-of-domain. Its matching quality continues to improve with additional test-time search, allowing compute to be traded for match quality. In pairwise listening tests, listeners preferred Synth-JEPA in 85% of trials overall. Together, these results show that an audio representation with a parameter-induced geometry allows synthesizer sound matching to be approached as an effective renderer-free search problem.
Abstract:Sound matching can be formulated as optimizing synthesizer parameters against an audio-domain objective. However, objectives derived from generic audio representations are often difficult to optimize, while direct search requires rendering every candidate. We introduce Synth-JEPA, which learns mutually predictive audio and parameter representations from paired synthesizer data. At inference, candidate parameters are scored directly in this learned space, yielding a renderer-free objective whose audio geometry is shaped by parameter correspondences rather than generic audio similarity. We evaluate Synth-JEPA on Surge XT using held-out synthesizer sounds and out-of-domain NSynth and FSD50K targets, against inverse models, direct search, and learned proxy objectives. Synth-JEPA outperforms all baselines in-domain and remains competitive out-of-domain. Its matching quality continues to improve with additional test-time search, allowing compute to be traded for match quality. In pairwise listening tests, listeners preferred Synth-JEPA in 85% of trials overall. Together, these results show that an audio representation with a parameter-induced geometry allows synthesizer sound matching to be approached as an effective renderer-free search problem.
Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
arXiv:2609.31024 [cs.SD]
(or arXiv:2609.31024v1 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2609.31024 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 265 words
Related coverage
Hamiltonian JEPA: Action-Conditioned World Models with an Inherited Control State
arXiv · Sep 30, 2026
Hierarchical Self-Supervised Representation Learning Framework for Multivariate Time Series Grounded in ECG Analysis
arXiv · Jul 24, 2026
Flow-JEPA: Flow Matching for Robust Latent Dynamics in JEPA World Models
arXiv · Sep 1, 2026
4DGS-JEPA: Temporally Compositional Joint-Embedding Prediction for Dynamic Gaussian Splatting
arXiv · Sep 24, 2026