aiqbench analysis
This research identifies structural and rhetorical patterns as more reliable predictors for cross-lingual transfer in scientific text classification than linguistic proximity alone. By leveraging these discourse-level regularities, the proposed methods achieve parity with encoder baselines in-domain and outperform them when transferring to unseen languages. The findings offer a robust framework for improving accessibility of multilingual digital libraries without relying on extensive non-English training data.
Improving Cross-Lingual Transfer for Sequential Sentence Classification in Research Papers via Structural Similarity
arXiv:2609.19650v1 Announce Type: new Abstract: Sequential sentence classification (SSC) is an essential task for structuring scientific publications, and extending SSC research to languages other than English can improve accessibility to scientific knowledge in multilingual digital libraries. Cross-lingual transfer is a promising approach to address the scarcity of training data in non-English languages. Prior work on other natural language processing tasks has shown the benefits of capturing linguistic similarity between source and target languages. However, SSC inherently depends on patterns at the discourse level, such as label sequences and positional regularities, which appear consistently across languages regardless of linguistic differences. To examine the factors that determine transfer success in SSC, we constructed a multilingual SSC dataset covering 13 non-English languages collected from five academic databases. Our cross-lingual transfer experiments, using both encoder-based and generative models, show that linguistic proximity has no consistent predictive power for transfer performance, whereas structural similarity in rhetorical organization shows a weak but consistent positive correlation across models. After controlling for source-language performance, the similarity of label distributions is the most consistent predictor. Building on this finding, we propose a set of three methods that explicitly leverage structural information using generative models. In the in-domain evaluation, the best combination reaches parity with the strongest encoder baselines, and in transfer to languages unseen during training, it outperforms the strongest encoder baseline.
Abstract:Sequential sentence classification (SSC) is an essential task for structuring scientific publications, and extending SSC research to languages other than English can improve accessibility to scientific knowledge in multilingual digital libraries. Cross-lingual transfer is a promising approach to address the scarcity of training data in non-English languages. Prior work on other natural language processing tasks has shown the benefits of capturing linguistic similarity between source and target languages. However, SSC inherently depends on patterns at the discourse level, such as label sequences and positional regularities, which appear consistently across languages regardless of linguistic differences. To examine the factors that determine transfer success in SSC, we constructed a multilingual SSC dataset covering 13 non-English languages collected from five academic databases. Our cross-lingual transfer experiments, using both encoder-based and generative models, show that linguistic proximity has no consistent predictive power for transfer performance, whereas structural similarity in rhetorical organization shows a weak but consistent positive correlation across models. After controlling for source-language performance, the similarity of label distributions is the most consistent predictor. Building on this finding, we propose a set of three methods that explicitly leverage structural information using generative models. In the in-domain evaluation, the best combination reaches parity with the strongest encoder baselines, and in transfer to languages unseen during training, it outperforms the strongest encoder baseline.
Accepted at JCDL 2026 (ACM/IEEE Joint Conference on Digital Libraries), Frisco, TX, USA, October 13-16, 2026. 12 pages, 5 figures, 9 tables. DOI: https://doi.org/10.1145/3805696.3846040
Computation and Language (cs.CL); Digital Libraries (cs.DL)
arXiv:2609.19650 [cs.CL]
(or arXiv:2609.19650v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.19650 arXiv-issued DOI via DataCite (pending registration)
https://doi.org/10.1145/3805696.3846040 DOI(s) linking to related resources
Full text under the publisher's open licence · 270 words
Related coverage
Why Better Cross-Lingual Alignment Fails for Better Cross-Lingual Transfer: Case of Encoders
arXiv · Sep 28, 2026
Granite Embedding Multilingual R2: Open Apache 2.0 Multilingual Embeddings with 32K Context — Best Sub-100M Retrieval Quality
Hugging Face · May 14, 2026
Improving Cross-Lingual Token Representations by Adding a Pinch of SALT
arXiv · Sep 11, 2026
Parallel Tokenizers: Rethinking Encoder Models' Vocabulary Design in Cross-Lingual Transfer of Low-Resource Languages
arXiv · Jul 28, 2026