aiqbench analysis
SMEBench offers a controlled benchmark for neural music editing, where each example pairs a source, an instruction, and a target edit rendered from symbolic MIDI. That pairing lets researchers test whether a system preserves unchanged content and applies the requested change as separate properties, which many existing evaluations cannot do. The authors also show that common audio-editing metrics can reverse the ranking of systems, so readers comparing audio generation or editing models should check which metric a result depends on.
SMEBENCH: Benchmarking Neural Music Editing with Synthetized MIDI Edits
arXiv:2610.36715v1 Announce Type: cross Abstract: Neural music editing requires a system to perform a requested change while preserving the musical content that should remain unchanged. Existing evaluations often lack a paired target edit, making these requirements difficult to measure separately. We introduce SMEBench, a large-scale benchmark of controlled music edits constructed by modifying symbolic MIDI performances and rendering matched source and target audio. Each example contains a source, an instruction, a paired target, edit metadata, and complementary descriptions. The benchmark covers twelve instrumentation, articulation, dynamics, and pitch operations and includes a balanced 3,600-example test set together with a smaller real-audio complement derived from multitrack recordings. The paired targets allow source preservation, target agreement, and textual compatibility to be evaluated independently. We use these references to diagnose common audio-editing metrics and benchmark six representative editing systems, showing that the metric used to rank a system can materially change the conclusion. The dataset is available at our repository.
Abstract:Neural music editing requires a system to perform a requested change while preserving the musical content that should remain unchanged. Existing evaluations often lack a paired target edit, making these requirements difficult to measure separately. We introduce SMEBench, a large-scale benchmark of controlled music edits constructed by modifying symbolic MIDI performances and rendering matched source and target audio. Each example contains a source, an instruction, a paired target, edit metadata, and complementary descriptions. The benchmark covers twelve instrumentation, articulation, dynamics, and pitch operations and includes a balanced 3,600-example test set together with a smaller real-audio complement derived from multitrack recordings. The paired targets allow source preservation, target agreement, and textual compatibility to be evaluated independently. We use these references to diagnose common audio-editing metrics and benchmark six representative editing systems, showing that the metric used to rank a system can materially change the conclusion. The dataset is available at our repository.
Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)
arXiv:2610.36715 [cs.SD]
(or arXiv:2610.36715v1 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2610.36715 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 252 words
Related coverage
Topological Ignorability for Structural Causal Effects Beyond Means
arXiv · Jun 3, 2026
CausalProfiler: Generating Synthetic Benchmarks for Rigorous and Transparent Evaluation of Causal Machine Learning
arXiv · Aug 21, 2026
SyntheticHLS: Building Diverse Synthetic High-Level Synthesis Datasets using LLMs
arXiv · Oct 2, 2026
Psych-ECA: A Reproducible Semi-Synthetic Benchmark for Synthetic Control Arms in Longitudinal Psychiatry
arXiv · Jul 31, 2026