Skip to content
aiqbench takeMachine Learning

aiqbench analysis

This research addresses the specific computational challenge of Khmer language processing by unifying text recognition and word segmentation into a single model, thereby eliminating latency associated with sequential pipelines. By leveraging CTC decoders for parallel decoding, the framework improves efficiency for large-scale document processing without requiring external segmentation steps. This advancement is significant for optimizing retrieval-augmented generation (RAG) systems that handle low-resource languages lacking visible word delimiters.
Source coveringarXiv
What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

Towards a Joint Khmer Text Recognition and Word Segmentation

arXiv2 min read

arXiv:2608.30213v1 Announce Type: cross Abstract: Text recognition, or extracting electronic text from document images, has been indispensable for knowledge retrieval tasks, such as retrieval-augmented generation (RAG). For Khmer, extracted text is subject to an extra word segmentation step, as Khmer does not use any visible word delimiters to denote word boundaries. Thus, a recognition-then-segmentation pipeline for Khmer requires two separate sequential models; this is not only error-prone but also adds significant latency for large-scale document processing. This paper proposes a novel joint Khmer text recognition and word segmentation framework in a unified model. The proposed model, using a connectionist-temporal-classification (CTC) decoder for fast, parallel decoding, can be instructed to recognize Khmer text with ($b=1$) and without ($b=0$) word segmentation. Experimental results on different benchmark datasets of different document modalities (document, scene, and handwritten images) show that the proposed model can not only recognize characters in document images but also locate word boundaries, removing the need for an extra word segmentation step in a conventional sequential pipeline.

Abstract:Text recognition, or extracting electronic text from document images, has been indispensable for knowledge retrieval tasks, such as retrieval-augmented generation (RAG). For Khmer, extracted text is subject to an extra word segmentation step, as Khmer does not use any visible word delimiters to denote word boundaries. Thus, a recognition-then-segmentation pipeline for Khmer requires two separate sequential models; this is not only error-prone but also adds significant latency for large-scale document processing. This paper proposes a novel joint Khmer text recognition and word segmentation framework in a unified model. The proposed model, using a connectionist-temporal-classification (CTC) decoder for fast, parallel decoding, can be instructed to recognize Khmer text with ($b=1$) and without ($b=0$) word segmentation. Experimental results on different benchmark datasets of different document modalities (document, scene, and handwritten images) show that the proposed model can not only recognize characters in document images but also locate word boundaries, removing the need for an extra word segmentation step in a conventional sequential pipeline.

Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)

arXiv:2608.30213 [cs.CV]

(or arXiv:2608.30213v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2608.30213 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Full text under the publisher's open licence · 262 words

Related coverage