analiza aiqbench
G-Loss introduces a graph-guided approach to fine-tuning language models that leverages global semantic structures via document-similarity graphs. By incorporating semi-supervised label propagation into the loss function, this method achieves faster convergence and improved classification accuracy across standard benchmarks compared to traditional local neighborhood losses. This research advances machine learning techniques by addressing limitations in how current embedding spaces capture broad semantic relationships during fine-tuning.
G-Loss: Graph-Guided Fine-Tuning of Language Models
arXiv:2604.25853v3 Announce Type: replace Abstract: Traditional loss functions, including cross-entropy, contrastive, triplet, and su pervised contrastive losses, used for fine-tuning pre-trained language models such as BERT, operate only within local neighborhoods and fail to account for the global semantic structure. We present G-Loss, a graph-guided loss function that incorporates semi-supervised label propagation to use structural relationships within the embedding manifold. G-Loss builds a document-similarity graph that captures global semantic relationships, thereby guiding the model to learn more discriminative and robust embeddings. We evaluate G-Loss on five benchmark datasets covering key downstream classification tasks: MR (sentiment analysis), R8 and R52 (topic categorization), Ohsumed (medical document classification), and 20NG (news categorization). In the majority of experimental setups, G-Loss converges faster and produces semantically coherent embedding spaces, resulting in higher classification accuracy than models fine-tuned with traditional loss functions.
Abstract:Traditional loss functions, including cross-entropy, contrastive, triplet, and su pervised contrastive losses, used for fine-tuning pre-trained language models such as BERT, operate only within local neighborhoods and fail to account for the global semantic structure. We present G-Loss, a graph-guided loss function that incorporates semi-supervised label propagation to use structural relationships within the embedding manifold. G-Loss builds a document-similarity graph that captures global semantic relationships, thereby guiding the model to learn more discriminative and robust embeddings. We evaluate G-Loss on five benchmark datasets covering key downstream classification tasks: MR (sentiment analysis), R8 and R52 (topic categorization), Ohsumed (medical document classification), and 20NG (news categorization). In the majority of experimental setups, G-Loss converges faster and produces semantically coherent embedding spaces, resulting in higher classification accuracy than models fine-tuned with traditional loss functions.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2604.25853 [cs.CL]
(or arXiv:2604.25853v3 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 149 cuvinte
Subiecte conexe
G-Loss: Graph-Guided Fine-Tuning of Language Models
arXiv · 1 sept. 2026
PIRAMID: Progress and Plans
LessWrong · 27 iul. 2026
Data-Centric Post-Training for Financial Reasoning: Mining, Distillation, and Verifiable Learning
arXiv · 10 sept. 2026
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · 18 sept. 2026