aiqbench analysis
This paper introduces ConstructCIE, a specialized dataset designed to extract implicit causal factors from construction accident narratives using hierarchical schemas and LLMs. The study reveals that while current models can broadly identify accident types, they struggle with precise span-level evidence extraction due to challenges in domain grounding and boundary detection. These findings highlight critical gaps for advancing NLP systems capable of reliably parsing complex, distributed textual evidence in safety-critical domains.
ConstructCIE: A Dataset for Extracting Causal Information from Construction Accident Narratives
arXiv:2608.06495v2 Announce Type: replace Abstract: Construction accident narratives contain rich causal information, but the evidence is often implicit, long-span, and distributed. We introduce ConstructCIE, a manually annotated dataset for Causal Information Extraction from OSHA construction accident reports. The dataset uses a hierarchical schema for accident types, causal factors, sub-causal factors, and supporting evidence spans. We evaluate supervised sequence taggers and instruction-tuned LLMs in an end-to-end hierarchical extraction setting. Results show that most evaluated models achieve strong accident-type prediction and recover broad causal meaning but remain limited in precise span-level extraction. Joint Hierarchical Extraction generally achieves stronger exact and soft matching, while Individual Hierarchical Extraction sometimes achieves higher keyword F1. Error distributions vary by extraction strategy, but evidence-selection and span-boundary errors remain common. These findings show that reliable Causal Information Extraction for construction accidents requires stronger domain grounding and more accurate evidence extraction.
Abstract:Construction accident narratives contain rich causal information, but the evidence is often implicit, long-span, and distributed. We introduce ConstructCIE, a manually annotated dataset for Causal Information Extraction from OSHA construction accident reports. The dataset uses a hierarchical schema for accident types, causal factors, sub-causal factors, and supporting evidence spans. We evaluate supervised sequence taggers and instruction-tuned LLMs in an end-to-end hierarchical extraction setting. Results show that most evaluated models achieve strong accident-type prediction and recover broad causal meaning but remain limited in precise span-level extraction. Joint Hierarchical Extraction generally achieves stronger exact and soft matching, while Individual Hierarchical Extraction sometimes achieves higher keyword F1. Error distributions vary by extraction strategy, but evidence-selection and span-boundary errors remain common. These findings show that reliable Causal Information Extraction for construction accidents requires stronger domain grounding and more accurate evidence extraction.
Computation and Language (cs.CL)
arXiv:2608.06495 [cs.CL]
(or arXiv:2608.06495v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.06495 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 230 words
Related coverage
Does Your LLM Trust You?
LessWrong · Aug 4, 2026

Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
METR · Aug 26, 2026
Some Intuitions on Steering Vectors
LessWrong · Sep 29, 2026
Structure Over Scale: Schema-Constrained Causal Graphs for RAG
arXiv · Jul 28, 2026