aiqbench analysis
This research introduces TAB-PO, a novel preference optimization method designed specifically for token-critical structured generation tasks like JSON schema validation. By addressing gradient dilution and token erosion inherent in standard DPO approaches, the technique significantly improves semantic labeling metrics using Llama and Qwen models without compromising textual grounding.
TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generation
arXiv:2603.00025v3 Announce Type: replace Abstract: Direct Preference Optimization (DPO) is effective for offline alignment but poorly matched to ontology-driven structured prediction, where preferred and rejected JSON often differ by only a few schema-defining tokens. In this low-edit-distance regime, sequence-level DPO spreads gradient mass across non-critical serialization tokens (gradient dilution) and can reduce likelihood on rare preferred schema tokens (token erosion). To address these limitations, we first develop a confusion-aware preference-construction strategy combining expert-curated ambiguity patterns with validation-set SFT structured-error modes, producing minimally perturbed, schema-valid negatives for ontology-level decision errors. We then introduce Token-Adaptive Barrier Preference Optimization (TAB-PO), a post-SFT objective for token-critical structured generation with a confidence-gated token-level barrier that anchors under-confident schema tokens. On SciERC, with Llama/Qwen models, TAB-PO improves ontology-critical semantic-label and relational-linking metrics by 11.59% relative to SFT, wins 100% of comparisons against strongest token-level/sequence-level DPO variants, and surpasses strongest frontier baselines on these metrics by 14.71% relative while improving textual grounding.
Abstract:Direct Preference Optimization (DPO) is effective for offline alignment but poorly matched to ontology-driven structured prediction, where preferred and rejected JSON often differ by only a few schema-defining tokens. In this low-edit-distance regime, sequence-level DPO spreads gradient mass across non-critical serialization tokens (gradient dilution) and can reduce likelihood on rare preferred schema tokens (token erosion). To address these limitations, we first develop a confusion-aware preference-construction strategy combining expert-curated ambiguity patterns with validation-set SFT structured-error modes, producing minimally perturbed, schema-valid negatives for ontology-level decision errors. We then introduce Token-Adaptive Barrier Preference Optimization (TAB-PO), a post-SFT objective for token-critical structured generation with a confidence-gated token-level barrier that anchors under-confident schema tokens. On SciERC, with Llama/Qwen models, TAB-PO improves ontology-critical semantic-label and relational-linking metrics by 11.59% relative to SFT, wins 100% of comparisons against strongest token-level/sequence-level DPO variants, and surpasses strongest frontier baselines on these metrics by 14.71% relative while improving textual grounding.
Computation and Language (cs.CL)
arXiv:2603.00025 [cs.CL]
(or arXiv:2603.00025v3 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2603.00025 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 243 words
Related coverage
TAB-PO: Preference Optimization with a Token-Level Adaptive Barrier for Token-Critical Structured Generation
arXiv · Jun 12, 2026
[AINews] High Taste Testers Ep. 1
Latent Space · Oct 17, 2026
From CUDA to MLX: How K-Search Brings Decades of Kernel Expertise to Apple Silicon
BAIR Blog · Jul 29, 2026
Amazon SageMaker Inference: 2026 year-to-date launches in review
AWS · Sep 18, 2026