aiqbench analysis
This preprint presents SHAKTI, a Hindi benchmark for telling human-written text from LLM output, and TRIDENT, a detector that checks whether text stays semantically stable after round-trip translation. The authors report that TRIDENT beats baselines and holds up across seven languages and unseen models, but they also flag sensitivity to the choice of translation model, which bears directly on how portable the method is. For readers, the useful takeaways are the benchmark's value for low-resource languages and the need to see independent replication before relying on detection accuracy.
TRIDENT and SHAKTI: A Framework and Benchmark for Hindi AI-Text Detection
arXiv:2610.41232v1 Announce Type: new Abstract: As Large Language Models (LLMs) become increasingly multilingual, distinguishing human-written from AI-generated text is essential for maintaining information integrity. However, research remains English-centric, with limited benchmarks for low-resource languages such as Hindi. To address this gap, we introduce SHAKTI, a large-scale Hindi benchmark covering four genres, eight LLM families, multiple prompting strategies, and adversarial transformations. We evaluate its quality through human annotation and analyze linguistic, grammatical, and sentiment-based differences between human- and AI-generated texts. Building on these findings, we propose TRIDENT, a round-trip translation-based detection framework with three variants: TRIDENT-S (static threshold), TRIDENT-D (dynamic threshold), and TRIDENT-C (logistic regression using multi-distance RTT features). TRIDENT leverages semantic consistency between original and round-trip translated texts to identify AI-generated content. Extensive experiments show that TRIDENT outperforms conventional and supervised baselines, with TRIDENT-S achieving the best overall performance. Further evaluations across seven languages, adversarial settings, and unseen LLMs demonstrate its multilingual applicability and robustness, while highlighting sensitivity to the translation model.
Abstract:As Large Language Models (LLMs) become increasingly multilingual, distinguishing human-written from AI-generated text is essential for maintaining information integrity. However, research remains English-centric, with limited benchmarks for low-resource languages such as Hindi. To address this gap, we introduce SHAKTI, a large-scale Hindi benchmark covering four genres, eight LLM families, multiple prompting strategies, and adversarial transformations. We evaluate its quality through human annotation and analyze linguistic, grammatical, and sentiment-based differences between human- and AI-generated texts. Building on these findings, we propose TRIDENT, a round-trip translation-based detection framework with three variants: TRIDENT-S (static threshold), TRIDENT-D (dynamic threshold), and TRIDENT-C (logistic regression using multi-distance RTT features). TRIDENT leverages semantic consistency between original and round-trip translated texts to identify AI-generated content. Extensive experiments show that TRIDENT outperforms conventional and supervised baselines, with TRIDENT-S achieving the best overall performance. Further evaluations across seven languages, adversarial settings, and unseen LLMs demonstrate its multilingual applicability and robustness, while highlighting sensitivity to the translation model.
Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2610.41232 [cs.AI]
(or arXiv:2610.41232v1 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2610.41232 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 255 words
Related coverage
A Measurement Perspective on Benchmarking LLM-Generated Text Detectors using Item Response Theory
arXiv · Oct 21, 2026
Benchmarking the Benchmarks: Evaluating Benchmarks for Conversational Agents
arXiv · Aug 7, 2026
A reproducible and extensible framework for benchmarking competing risks survival models
arXiv · Aug 4, 2026
KVoiceBench, KOpenAudioBench, and KMMAU: Agent-Driven Korean Speech Benchmarks for Evaluating SpeechLMs
arXiv · Sep 1, 2026