analiza aiqbench
BridgeAlign introduces a novel preference alignment pipeline specifically designed for the humanities and social sciences, addressing a critical gap where traditional LLM training focuses on verifiable facts rather than nuanced qualitative judgment. By synthesizing over 210k samples through controlled quality degradation and persona-based inversion, this method successfully improves Qwen3-8B performance across diverse benchmarks without sacrificing knowledge accuracy. This research advances the frontier of applying advanced alignment techniques to open-ended domains that require sophisticated understanding beyond simple correctness checks.
BridgeAlign: Bridging Preference Alignment for Humanities and Social Sciences
arXiv:2607.27366v1 Announce Type: new Abstract: While data synthesis for large language models (LLMs) is prevalent, it primarily targets domains with verifiable answers, overlooking open-ended humanities and social sciences (HSS), where nuanced quality judgments matter more than objective correctness. This makes preference alignment a natural paradigm for broad HSS tasks. Yet existing methods are either costly or not tailored to broad HSS disciplines. We thus propose BridgeAlign, among the first preference-alignment pipelines for broad HSS disciplines, with three phases: i) Seed Curation: curating HSS seed documents from web corpora via heuristic/LLM-based filtering and text refinement; ii) Preference Data Synthesis: generating preference triplets via persona-based instruction inversion with Q&A consistency checks; iii) Preference Optimization: moving beyond naive human-vs-model heuristics by first grounding preferences in HSS quality rubric, then generating transitional responses via controlled quality degradation to form near-boundary preference pairs for finer-grained quality discrimination. Aligning over 210k synthetic preference samples, BridgeAlign enables Qwen3-8B to achieve the best average across 17 benchmarks against 11 strong baselines; importantly, leading on both human-preference and knowledge-based capabilities at once, with no trade-off between them, as supported by extensive experiments and contextualized by existing theories.
Abstract:While data synthesis for large language models (LLMs) is prevalent, it primarily targets domains with verifiable answers, overlooking open-ended humanities and social sciences (HSS), where nuanced quality judgments matter more than objective correctness. This makes preference alignment a natural paradigm for broad HSS tasks. Yet existing methods are either costly or not tailored to broad HSS disciplines. We thus propose BridgeAlign, among the first preference-alignment pipelines for broad HSS disciplines, with three phases: i) Seed Curation: curating HSS seed documents from web corpora via heuristic/LLM-based filtering and text refinement; ii) Preference Data Synthesis: generating preference triplets via persona-based instruction inversion with Q&A consistency checks; iii) Preference Optimization: moving beyond naive human-vs-model heuristics by first grounding preferences in HSS quality rubric, then generating transitional responses via controlled quality degradation to form near-boundary preference pairs for finer-grained quality discrimination. Aligning over 210k synthetic preference samples, BridgeAlign enables Qwen3-8B to achieve the best average across 17 benchmarks against 11 strong baselines; importantly, leading on both human-preference and knowledge-based capabilities at once, with no trade-off between them, as supported by extensive experiments and contextualized by existing theories.
Computation and Language (cs.CL)
arXiv:2607.27366 [cs.CL]
(or arXiv:2607.27366v1 [cs.CL] for this version)
Text integral sub licența deschisă a publicației · 195 cuvinte
Subiecte conexe
Links #3: 2026/06 Part 1
LessWrong · 15 iun. 2026
State of the Field: AI for Epistemics and Coordination
LessWrong · 2 oct. 2026
From Imitation to Alignment: Human-Preference Flow Policies for Long-Horizon Sidewalk Navigation
arXiv · 12 iun. 2026
Eliciting ESG Preferences for Reinforcement Learning-Based Portfolio Optimization
arXiv · 3 sept. 2026