aiqbench analysis
This study addresses critical data scarcity in low-resource languages by creating a specialized Nepali FAQ dataset to enhance passport service retrieval systems. The research demonstrates that fine-tuned transformer models, particularly multilingual E5 embeddings, significantly outperform traditional BM25 baselines in semantic matching tasks. This work highlights the practical application of machine learning techniques to improve public sector information access where annotated training data is traditionally unavailable.
Nepali Passport Question Answering: A Low-Resource Dataset for Public Service Applications
arXiv:2603.13320v2 Announce Type: replace-cross Abstract: Nepali, a low-resource language, faces significant challenges in building an effective information retrieval system due to the unavailability of annotated data and computational linguistic resources. In this study, we attempt to address this gap by preparing a pair-structured Nepali Question-Answer dataset. We focus on Frequently Asked Questions (FAQs) for passport-related services, building a data set for training and evaluation of IR models. In our study, we have fine-tuned transformer-based embedding models for semantic similarity in question-answer retrieval. The fine-tuned models were compared with the baseline BM25. In addition, we implement a hybrid retrieval approach, integrating fine-tuned models with BM25, and evaluate the performance of the hybrid retrieval. Our results show that the fine-tuned SBERT-based models outperform BM25, whereas multilingual E5 embedding-based models achieve the highest retrieval performance among all evaluated models.
Abstract:Nepali, a low-resource language, faces significant challenges in building an effective information retrieval system due to the unavailability of annotated data and computational linguistic resources. In this study, we attempt to address this gap by preparing a pair-structured Nepali Question-Answer dataset. We focus on Frequently Asked Questions (FAQs) for passport-related services, building a data set for training and evaluation of IR models. In our study, we have fine-tuned transformer-based embedding models for semantic similarity in question-answer retrieval. The fine-tuned models were compared with the baseline BM25. In addition, we implement a hybrid retrieval approach, integrating fine-tuned models with BM25, and evaluate the performance of the hybrid retrieval. Our results show that the fine-tuned SBERT-based models outperform BM25, whereas multilingual E5 embedding-based models achieve the highest retrieval performance among all evaluated models.
7 pages, 3 figures, Accepted and presented at RegICON 2025 (Regional International Conference on Natural Language Processing): NLP for East India, North East India and Southeast Asia. this https URL
Information Retrieval (cs.IR); Machine Learning (cs.LG)
arXiv:2603.13320 [cs.IR]
(or arXiv:2603.13320v2 [cs.IR] for this version)
Full text under the publisher's open licence · 175 words
Related coverage
Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
METR · Aug 26, 2026
Frontier AI Risk Report (February–March 2026)
METR · May 19, 2026
Informe de riesgos de la IA de frontera (febrero–marzo de 2026)
METR · May 19, 2026
Frontier Risk Report (February to March 2026)
METR · May 19, 2026