Efficient Context-Limited Telescope Bibliography Classification for the WASP-2025 Shared Task Using SciBERT

Abstract:The creation of telescope bibliographies is a crucial part of assessing the scientific impact of observatories and ensuring reproducibility in astronomy. This task involves identifying, categorizing, and linking scientific publications that reference or use specific telescopes. However, this process remains largely manual and resource intensive. In this work, we present an efficient SciBERT-based approach for automatic classification of scientific papers into four categories - science, instrumentation, mention, and not telescope. Despite strict context-length constraints (maximum 512 tokens) and limited compute resources, our approach achieved a macro F1 score of 0.89, ranking at the top of the WASP-2025 leaderboard. We analyze the effect of truncation and show that even with half the samples exceeding the token limit, SciBERT's domain alignment enables robust classification. We discuss trade-offs between truncation, chunking, and long-context models, providing insights into the efficiency frontier for scientific text curation.

3 pages, 2 tables. 1st place system description for the TRACS shared task at WASP 2025 (Third Workshop for Artificial Intelligence for Scientific Publications), co-located with IJCNLP-AACL 2025. Published version: this https URL . Code: this https URL

Machine Learning (cs.LG); Instrumentation and Methods for Astrophysics (astro-ph.IM)

I.2.7; I.2.6; H.3.3; J.2

arXiv:2609.01647 [cs.LG]

(or arXiv:2609.01647v1 [cs.LG] for this version)

Proceedings of the Third Workshop for Artificial Intelligence for Scientific Publications (WASP 2025), pages 192-194, Mumbai, India and virtual, December 2025. Association for Computational Linguistics

https://doi.org/10.18653/v1/2025.wasp-main.21 DOI(s) linking to related resources