aiqbench analysis
This study challenges the assumption that sharing parameters or priors in multi-schema motion VAEs guarantees directly interchangeable latent representations. The research demonstrates that while correspondence can be recovered through linear calibration, direct compatibility requires specific paired training data to achieve high token agreement and recall. These findings are critical for developers building unified generative models, as they highlight the necessity of explicit alignment strategies rather than relying solely on architectural sharing.
What Makes a Latent Space Shared? Direct and Recoverable Compatibility in Motion VAEs
arXiv:2610.17376v1 Announce Type: new Abstract: A shared latent space is often assumed to make representations from different schemas directly interchangeable, yet sharing parameters, dimensionality, or a prior does not determine how corresponding instances are expressed in latent coordinates. We show that this assumption conflates recoverable correspondence, direct compatibility, and decoder reuse. In a controlled multi-schema motion VAE on HumanML3D, even identical inputs yield only $5.9$--$17.7\%$ raw Recall@1 and $0.221$--$0.484$ paired-token cosine, despite correspondence being almost perfectly recoverable after linear calibration. Cycle consistency strengthens recoverability but still leaves direct retrieval and token agreement near zero. In contrast, only $64$ paired clips raise raw Recall@1 to $88.4$--$90.3\%$ and token cosine to approximately $0.82$, making most post-hoc alignment unnecessary, although crossed decoding remains substantially worse than native reconstruction. We further find that reducing measured marginal discrepancy does not imply schema invariance. These results show that a shared backbone or prior does not ensure directly compatible latent coordinates, while limited paired correspondence can substantially improve direct compatibility in the tested setting.
Title:What Makes a Latent Space Shared? Direct and Recoverable Compatibility in Motion VAEs
Abstract:A shared latent space is often assumed to make representations from different schemas directly interchangeable, yet sharing parameters, dimensionality, or a prior does not determine how corresponding instances are expressed in latent coordinates. We show that this assumption conflates recoverable correspondence, direct compatibility, and decoder reuse. In a controlled multi-schema motion VAE on HumanML3D, even identical inputs yield only $5.9$--$17.7\%$ raw Recall@1 and $0.221$--$0.484$ paired-token cosine, despite correspondence being almost perfectly recoverable after linear calibration. Cycle consistency strengthens recoverability but still leaves direct retrieval and token agreement near zero. In contrast, only $64$ paired clips raise raw Recall@1 to $88.4$--$90.3\%$ and token cosine to approximately $0.82$, making most post-hoc alignment unnecessary, although crossed decoding remains substantially worse than native reconstruction. We further find that reducing measured marginal discrepancy does not imply schema invariance. These results show that a shared backbone or prior does not ensure directly compatible latent coordinates, while limited paired correspondence can substantially improve direct compatibility in the tested setting.
Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2610.17376 [cs.AI]
(or arXiv:2610.17376v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.17376 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 275 words
Related coverage
[AINews] High Taste Testers Ep. 1
Latent Space · Oct 17, 2026
[AINews] not much happened today
Latent Space · Sep 18, 2026
[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · Sep 17, 2026
[AINews] Meta Muse Wonder & Wonder Full: MSL’s Claude 5.5-tier Frontier Model
Latent Space · Oct 13, 2026