aiqbench analysis
HukukBERT addresses a critical gap in LegalTech by introducing the first high-volume, domain-specific language model trained exclusively on Turkish legal corpus data. The researchers achieved superior performance over existing baselines using a hybrid pre-training strategy that mitigates train-test contamination through synthetic benchmarking. This release provides essential infrastructure for advancing NLP applications like judgment prediction and document classification within the Turkish judicial system.
HUKUKBERT: Domain-Specific Language Model for Turkish Law
arXiv:2604.04790v2 Announce Type: replace Abstract: Natural language processing (NLP) advances have powered a generation of LegalTech systems, but Turkish law remains under-served by domain-specific data and models. English has legal encoders such as LEGAL-BERT; no comparable high-volume Turkish counterpart exists. We introduce HukukBERT, a Turkish legal language model trained on a 19 GB cleaned corpus using a hybrid domain-adaptive pre-training (DAPT) recipe that mixes Whole-Word Masking, Token Span Masking, Word Span Masking, and targeted Keyword Masking. We compared our 48K WordPiece tokenizer and DAPT pipeline against general-purpose and existing domain-specific Turkish models. On the Legal Cloze Test - a masked legal term prediction benchmark over Turkish court decisions - HukukBERT reaches 84.40% Top-1 accuracy and beats every baseline we tested. The Legal Cloze Test is synthetically constructed, so its passages are absent from the pre-training corpus by construction, eliminating train-test contamination. On the downstream task of structural segmentation of official Turkish court decisions, it reaches a 92.8% document pass rate. We release HukukBERT to support Turkish legal NLP work in named entity recognition, judgment prediction, and document classification.
Abstract:Natural language processing (NLP) advances have powered a generation of LegalTech systems, but Turkish law remains under-served by domain-specific data and models. English has legal encoders such as LEGAL-BERT; no comparable high-volume Turkish counterpart exists. We introduce HukukBERT, a Turkish legal language model trained on a 19 GB cleaned corpus using a hybrid domain-adaptive pre-training (DAPT) recipe that mixes Whole-Word Masking, Token Span Masking, Word Span Masking, and targeted Keyword Masking. We compared our 48K WordPiece tokenizer and DAPT pipeline against general-purpose and existing domain-specific Turkish models. On the Legal Cloze Test - a masked legal term prediction benchmark over Turkish court decisions - HukukBERT reaches 84.40% Top-1 accuracy and beats every baseline we tested. The Legal Cloze Test is synthetically constructed, so its passages are absent from the pre-training corpus by construction, eliminating train-test contamination. On the downstream task of structural segmentation of official Turkish court decisions, it reaches a 92.8% document pass rate. We release HukukBERT to support Turkish legal NLP work in named entity recognition, judgment prediction, and document classification.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2604.04790 [cs.CL]
(or arXiv:2604.04790v2 [cs.CL] for this version)
Öztürk, M. U., Türkoğlu, T., & Buz-Yalug, B. (2026). HukukBERT: Domain-Specific Language Model for Turkish Law. Journal of Computational Law and Legal Technology, 1-15
https://doi.org/10.47852/bonviewJCLLT620210346 DOI(s) linking to related resources
Full text under the publisher's open licence · 218 words
Related coverage
Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification
arXiv · Jun 8, 2026
How to Fine-Tune Nemotron 3.5 ASR for Your Language, Domain, or Accent
Hugging Face · Jun 4, 2026
From Tweets to Trades: Analyzing the Influence of Public Mood over Stock Market Performance in Turkiye
arXiv · Oct 1, 2026
Why AI Needs a “Genie Coefficient”
IEEE Spectrum · Jul 21, 2026