aiqbench analysis
This paper builds a gold-standard annotated dataset of Nepali political texts and benchmarks classical ML, deep learning (fastText-LSTM scored highest, 0.913 macro F1), and few-shot LLMs (Qwen3-8B, 0.848 macro F1) on extracting election commitments. The results are useful as a low-resource NLP baseline, but they come from a single language and a dataset whose annotation agreement (kappa 0.777) is moderate, so generalization claims should be read cautiously. The accountability application is a reasonable use case, but the paper's technical value lies mainly in the dataset and the model comparison.
From Promises to Accountability: Comparative AI Modeling and Visualization of Electoral Commitments in Nepali Political Texts
arXiv:2610.32150v1 Announce Type: new Abstract: Election Commitment Mining refers to the analysis of election commitments made by electoral candidates or political parties during election campaigns. However, many elected candidates and political parties, especially in developing South Asian countries, do not fully deliver on the promises they make before elections. Even after the elections, commitments presented through election manifestos, posters, pamphlets, videos, and other campaign materials are often not systematically or transparently archived. As a result, these promises tend to fade away over time, making it difficult to hold the elected candidates accountable for their commitments. To address this persistent issue, this research aims to develop a system capable of structuring the commitments made by candidates during election periods. In this work, we created the first gold-standard dataset of Nepali political texts (Cohen's kappa = 0.777) and evaluated machine learning, deep learning and large language models to automatically extract and analyze election commitments. LaBSE-LR achieved the best results among machine learning models (0.897 accuracy, 0.896 macro F1-score, 0.947 AUROC), while fastText-LSTM achieved the highest performance among deep learning models (0.914 accuracy, 0.913 macro F1-score, 0.961 AUROC). Among the three evaluated large language models, Qwen3-8B performed best in the few-shot setting (0.848 macro F1-score). The outcome of this work is integrated into a mobile application, where commitments can be visualized with the support of GAI, enhancing transparency, civic engagement, and electoral accountability.
Abstract:Election Commitment Mining refers to the analysis of election commitments made by electoral candidates or political parties during election campaigns. However, many elected candidates and political parties, especially in developing South Asian countries, do not fully deliver on the promises they make before elections. Even after the elections, commitments presented through election manifestos, posters, pamphlets, videos, and other campaign materials are often not systematically or transparently archived. As a result, these promises tend to fade away over time, making it difficult to hold the elected candidates accountable for their commitments. To address this persistent issue, this research aims to develop a system capable of structuring the commitments made by candidates during election periods. In this work, we created the first gold-standard dataset of Nepali political texts (Cohen's kappa = 0.777) and evaluated machine learning, deep learning and large language models to automatically extract and analyze election commitments. LaBSE-LR achieved the best results among machine learning models (0.897 accuracy, 0.896 macro F1-score, 0.947 AUROC), while fastText-LSTM achieved the highest performance among deep learning models (0.914 accuracy, 0.913 macro F1-score, 0.961 AUROC). Among the three evaluated large language models, Qwen3-8B performed best in the few-shot setting (0.848 macro F1-score). The outcome of this work is integrated into a mobile application, where commitments can be visualized with the support of GAI, enhancing transparency, civic engagement, and electoral accountability.
Under review at Natural Language Processing (Cambridge University Press); revision submitted. 34 pages, 5 figures, 13 tables
Computation and Language (cs.CL)
arXiv:2610.32150 [cs.CL]
(or arXiv:2610.32150v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.32150 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 337 words
Related coverage
Improving Fake Profile Detection Through Multimodal Learning and Research-Guided Feature Integration
arXiv · Oct 21, 2026
Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Models
arXiv · Sep 29, 2026
See Selectively, Act Adaptively: Dual-Level Structural Decomposition for Bimanual Robot Manipulation
arXiv · Jun 12, 2026
CARLAR Influence: A Structured Hallucination Risk in Multimodal Cheque Extraction and an Exploratory Compressed-Image Countermeasure
arXiv · Oct 21, 2026