aiqbench analysis
The release of LEMMA-RCA addresses a critical bottleneck in AI development by providing the first large-scale, open-source dataset specifically tailored for root cause analysis across IT and OT systems. This multi-modal resource enables fair benchmarking of machine learning models in complex fault scenarios involving microservices, water distribution, and treatment facilities. By standardizing evaluation metrics for offline and online RCA tasks, this dataset will significantly accelerate the creation of more robust diagnostic algorithms.
LEMMA-RCA: A Large Multi-modal Multi-domain Dataset for Root Cause Analysis
arXiv:2406.05375v4 Announce Type: replace Abstract: Root cause analysis (RCA) is crucial for enhancing the reliability and performance of complex systems. However, progress in this field has been hindered by the lack of large-scale, open-source datasets tailored for RCA. To bridge this gap, we introduce LEMMA-RCA, a large dataset designed for diverse RCA tasks across multiple domains and modalities. LEMMA-RCA features various real-world fault scenarios from Information Technology (IT) and Operational Technology (OT) systems, including microservices, water distribution, and water treatment systems, with hundreds of system entities involved. We evaluate the performance of six baseline methods on LEMMA-RCA across various settings, including offline and online models, as well as single and multi-modal settings. Our study demonstrates the utility of LEMMA-RCA in facilitating fair evaluation and promoting the development of more robust RCA techniques. The dataset is publicly available at https://www.lemmarca.info, with archival metadata and access instructions deposited under DOI: https://doi.org/10.5281/zenodo.20516735.
Abstract:Root cause analysis (RCA) is crucial for enhancing the reliability and performance of complex systems. However, progress in this field has been hindered by the lack of large-scale, open-source datasets tailored for RCA. To bridge this gap, we introduce LEMMA-RCA, a large dataset designed for diverse RCA tasks across multiple domains and modalities. LEMMA-RCA features various real-world fault scenarios from Information Technology (IT) and Operational Technology (OT) systems, including microservices, water distribution, and water treatment systems, with hundreds of system entities involved. We evaluate the performance of six baseline methods on LEMMA-RCA across various settings, including offline and online models, as well as single and multi-modal settings. Our study demonstrates the utility of LEMMA-RCA in facilitating fair evaluation and promoting the development of more robust RCA techniques. The dataset is publicly available at this https URL, with archival metadata and access instructions deposited under DOI: this https URL.
Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2406.05375 [cs.AI]
(or arXiv:2406.05375v4 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2406.05375 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 243 words
Related coverage

Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
AWS · Sep 4, 2026

Presentation: From AI Agent Demo to Production: Automated Testing and Evaluation
InfoQ · Sep 7, 2026

Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
METR · Aug 26, 2026
Multi-Modal Generative Fuzzy System: Fuzzy Inference Guided Large Model Interactive Question Answering Framework
arXiv · Aug 18, 2026