aiqbench analysis
This arXiv preprint proposes a lightweight ensemble of LLMs that uses JSON-structured outputs and anonymous peer-review ranking to extract causal and other relations from popular science text and build HTML causal graphs. Its main evidence is a user study reporting 0.778 perceived accuracy against a human gold standard, which is a modest, subjective metric rather than a standard benchmark. Readers should note that the claims about reduced hallucination and single-model failure are not independently tested against stronger baselines, though the released open-license framework makes replication possible.
ReX-GG: a LLM Ensemble Pipeline for Relation-extraction and Graph Generation
arXiv:2610.41901v1 Announce Type: new Abstract: Current LLM ensemble frameworks focus on multi-step setups with additional modules for answer ranking, often opting for token and span analysis rather than structured outputs, leading to heavyweight architectures with potential fail states along the pipeline. Faster, lighter solutions are more vulnerable to hallucination propagation and can lack output control in more complex pipelines. This paper proposes a customisable, lightweight ensemble workflow of coordinated Large Language Models that leverages JSON-structured outputs and anonymous peer-review ranking to mitigate hallucinatory outputs and single-model failure points. The pipeline is demonstrated on a relation extraction task applied to popular science articles in English, targeting four ontologically-grounded relation types (strong causation, weak causation, contrastive, and compositional), with semantic node canonicalisation and interactive, colour-coded HTML causal graphs as the final output. Performance is evaluated through an anonymous user study, achieving an average perceived accuracy of 0.778 against a human-annotated gold standard. The modular architecture supports flexible deployment across both API-based and in-house LLM setups, and the full framework is released under an open license to foster reproducibility and collaborative research.
Abstract:Current LLM ensemble frameworks focus on multi-step setups with additional modules for answer ranking, often opting for token and span analysis rather than structured outputs, leading to heavyweight architectures with potential fail states along the pipeline. Faster, lighter solutions are more vulnerable to hallucination propagation and can lack output control in more complex pipelines. This paper proposes a customisable, lightweight ensemble workflow of coordinated Large Language Models that leverages JSON-structured outputs and anonymous peer-review ranking to mitigate hallucinatory outputs and single-model failure points. The pipeline is demonstrated on a relation extraction task applied to popular science articles in English, targeting four ontologically-grounded relation types (strong causation, weak causation, contrastive, and compositional), with semantic node canonicalisation and interactive, colour-coded HTML causal graphs as the final output. Performance is evaluated through an anonymous user study, achieving an average perceived accuracy of 0.778 against a human-annotated gold standard. The modular architecture supports flexible deployment across both API-based and in-house LLM setups, and the full framework is released under an open license to foster reproducibility and collaborative research.
Artificial Intelligence (cs.AI)
arXiv:2610.41901 [cs.AI]
(or arXiv:2610.41901v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.41901 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
Proceedings of the Knowledge Graphs and Large Language Models Workshop (KG-LLM) @ LREC26, Palma, Mallorca, Spain, 2026, pp. 93-101
https://doi.org/10.63317/2qkh8idwebsv Focus to learn more DOI(s) linking to related resources
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 297 words