aiqbench analysis
This article introduces "New York Smells," a significant multimodal dataset designed to bridge the gap between visual and olfactory data, addressing a critical bottleneck in machine perception research. By leveraging 7,000 paired smell-image pairs collected from natural environments, it enables advanced cross-modal learning tasks that were previously inaccessible due to a lack of diverse training signals. The work demonstrates how integrating sensory modalities can enhance representation learning for machines, moving beyond traditional hand-crafted features toward more robust AI models capable of processing complex chemical inputs alongside visual ones.
New York Smells: A Large Multimodal Dataset for Olfaction
arXiv:2511.20544v2 Announce Type: replace-cross Abstract: While olfaction is central to how animals perceive the world, this rich chemical sensory modality remains largely inaccessible to machines. One key bottleneck is the lack of diverse, multimodal olfactory training data collected in natural settings. We present New York Smells, a large dataset of paired image and olfactory signals captured ``in the wild.'' Our dataset contains 7,000 smell-image pairs from 3,500 distinct objects across indoor and outdoor environments, with approximately 70$\times$ more objects than existing olfactory datasets. Our benchmark has three tasks: cross-modal smell-to-image retrieval, recognizing scenes, objects, and materials from smell alone, and fine-grained discrimination between grass species. Through experiments on our dataset, we find that visual data enables cross-modal olfactory representation learning, and that our learned olfactory representations outperform widely-used hand-crafted features.
Abstract:While olfaction is central to how animals perceive the world, this rich chemical sensory modality remains largely inaccessible to machines. One key bottleneck is the lack of diverse, multimodal olfactory training data collected in natural settings. We present New York Smells, a large dataset of paired image and olfactory signals captured ``in the wild.'' Our dataset contains 7,000 smell-image pairs from 3,500 distinct objects across indoor and outdoor environments, with approximately 70$\times$ more objects than existing olfactory datasets. Our benchmark has three tasks: cross-modal smell-to-image retrieval, recognizing scenes, objects, and materials from smell alone, and fine-grained discrimination between grass species. Through experiments on our dataset, we find that visual data enables cross-modal olfactory representation learning, and that our learned olfactory representations outperform widely-used hand-crafted features.
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2511.20544 [cs.CV]
(or arXiv:2511.20544v2 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2511.20544 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 226 words
Related coverage
Multimodal Wearable-Based Olfactory-Induced Emotion Recognition in Arousal-Valence Dimensions
arXiv · Aug 5, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
Amazon is investing in the Lean Focused Research Organization
Amazon Science · Jul 26, 2026
ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Models
arXiv · Sep 30, 2026