aiqbench analysis
This study demonstrates a practical workflow for anonymizing Persian customer data by using large language models to generate training labels for a lightweight Named Entity Recognition model. The research highlights that specific open-source LLMs can effectively supervise compact classifiers, achieving high accuracy while reducing inference latency from massive H200 clusters to a single consumer GPU. This approach offers an efficient pathway for deploying privacy-preserving NLP tools in resource-constrained industrial environments without relying on heavy foundation models at inference time.
PersianAnonymizer: Evaluating LLM-Labeled Training for Efficient NER-based Anonymization in Persian
arXiv:2609.00958v1 Announce Type: new Abstract: We target practical anonymization of Persian customer chats by training a compact NER model from LLM-labeled supervision and selecting the best labeler for deployment. We compare three instruction-tuned LLMs: DeepSeek-V3-0324, GPT-OSS-120B, and Qwen3-235B-A22B-Instruct-2507, to produce span annotations under a shared JSON protocol, yielding four corpora (OSS_ZeroShot, Qwen_ZeroShot, Qwen_FewShot, DeepSeek_FewShot). A MatinaRoberta-based token-classifier is trained per corpus and evaluated with token-level Precision/Recall/F1 (overall and per-class). We also report Label Coverage Recall (LCR), the proportion of gold non-O tokens predicted as non-O, and quantify cross-labeler behavior via a token-level Venn on test annotations. Finally, we contrast test-set annotation latency of the LLMs on H200 nodes with the trained NER's test-time labeling on a single RTX 3090. Results show that supervision from OSS_ZeroShot yields the strongest macro-F1 and LCR, while the resulting NER labels an entire 40K-message test set in approximately 2 minutes on one consumer GPU. This establishes a practical path to high-quality, low-cost anonymization for Persian industrial data.
Abstract:We target practical anonymization of Persian customer chats by training a compact NER model from LLM-labeled supervision and selecting the best labeler for deployment. We compare three instruction-tuned LLMs: DeepSeek-V3-0324, GPT-OSS-120B, and Qwen3-235B-A22B-Instruct-2507, to produce span annotations under a shared JSON protocol, yielding four corpora (OSS_ZeroShot, Qwen_ZeroShot, Qwen_FewShot, DeepSeek_FewShot). A MatinaRoberta-based token-classifier is trained per corpus and evaluated with token-level Precision/Recall/F1 (overall and per-class). We also report Label Coverage Recall (LCR), the proportion of gold non-O tokens predicted as non-O, and quantify cross-labeler behavior via a token-level Venn on test annotations. Finally, we contrast test-set annotation latency of the LLMs on H200 nodes with the trained NER's test-time labeling on a single RTX 3090. Results show that supervision from OSS_ZeroShot yields the strongest macro-F1 and LCR, while the resulting NER labels an entire 40K-message test set in approximately 2 minutes on one consumer GPU. This establishes a practical path to high-quality, low-cost anonymization for Persian industrial data.
10 pages, 3 figures, 6 tables. Published at LREC 2026
Computation and Language (cs.CL)
arXiv:2609.00958 [cs.CL]
(or arXiv:2609.00958v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.00958 arXiv-issued DOI via DataCite (pending registration)
Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026), pages 4497-4506, 11-16 May 2026. ELRA Language Resources Association (ELRA), 2026
https://doi.org/10.63317/57u2ica9225o DOI(s) linking to related resources
Full text under the publisher's open licence · 214 words
Related coverage
Digital States 2026: AI Moves From Experiment to Infrastructure
govtech.com · Sep 24, 2026
A Study of ASR Adaptation and Representation Dimensionality Reduction in Persian Speech Emotion Recognition Using Whisper
arXiv · Aug 7, 2026
[AINews] not much happened today
Latent Space · Oct 3, 2026
[AINews] Opus 5.5 is good at explainer videos
Latent Space · Sep 29, 2026