aiqbench analysis
HSS-Synth introduces a novel data synthesis pipeline specifically designed to address the scarcity of high-quality training data in humanities and social sciences for large language models. By employing teacher-forced answering techniques and strict alignment checks, this method generates over 237k instruction-tuning samples that significantly outperform existing baselines while preserving domain integrity. The approach effectively mitigates hallucinations without sacrificing performance on standard benchmarks, offering a robust solution for expanding LLM capabilities in open-ended fields.
HSS-Synth: Humanities and Social Sciences Data Synthesis for LLMs
arXiv:2607.27379v1 Announce Type: new Abstract: High-quality, diverse data are vital for large language models (LLMs) but remain scarce and costly. Data synthesis is a viable alternative and succeeds on closed tasks, yet the humanities and social sciences (HSS) are overlooked, and their open-ended nature makes synthesis challenging. Moving beyond prior capability-centric, fragmented attempts, we adopt a subject-centric paradigm, define the first HSS domain system covering 14 mainstream fields, and introduce HSS-Synth, the first data synthesis pipeline for HSS. HSS-Synth comprises: (1) constructing seed documents from web corpora via multi-step filtering and text refinement evaluated by a judge; (2) specifying "requirements + persona" to backtranslate seed documents into diverse yet faithful instructions with a strict Q&A alignment check; and (3) breaking LLM response limits via teacher-forced Answering that feeds seed documents during response generation to anchor semantics, reduce hallucinations, and preserve tone and integrity. HSS-Synth yields 237k high-quality, diverse instruction-tuning samples that outperform 14 leading baselines on 16 benchmarks. The fine-tuned Qwen3-8B-Base sets a new SOTA and approaches the official Qwen3-8B, improving both human preference and knowledge capabilities without performance seesaws. Extensive experiments demonstrate HSS-Synth's robustness and transferability. Our code is publicly available at https://github.com/pengr/HSS-Synth.
Abstract:High-quality, diverse data are vital for large language models (LLMs) but remain scarce and costly. Data synthesis is a viable alternative and succeeds on closed tasks, yet the humanities and social sciences (HSS) are overlooked, and their open-ended nature makes synthesis challenging. Moving beyond prior capability-centric, fragmented attempts, we adopt a subject-centric paradigm, define the first HSS domain system covering 14 mainstream fields, and introduce HSS-Synth, the first data synthesis pipeline for HSS. HSS-Synth comprises: (1) constructing seed documents from web corpora via multi-step filtering and text refinement evaluated by a judge; (2) specifying "requirements + persona" to backtranslate seed documents into diverse yet faithful instructions with a strict Q&A alignment check; and (3) breaking LLM response limits via teacher-forced Answering that feeds seed documents during response generation to anchor semantics, reduce hallucinations, and preserve tone and integrity. HSS-Synth yields 237k high-quality, diverse instruction-tuning samples that outperform 14 leading baselines on 16 benchmarks. The fine-tuned Qwen3-8B-Base sets a new SOTA and approaches the official Qwen3-8B, improving both human preference and knowledge capabilities without performance seesaws. Extensive experiments demonstrate HSS-Synth's robustness and transferability. Our code is publicly available at this https URL.
Computation and Language (cs.CL)
arXiv:2607.27379 [cs.CL]
(or arXiv:2607.27379v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2607.27379 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 210 words
Related coverage
Links #3: 2026/06 Part 1
LessWrong · Jun 15, 2026
BridgeAlign: Bridging Preference Alignment for Humanities and Social Sciences
arXiv · Jul 31, 2026
State of the Field: AI for Epistemics and Coordination
LessWrong · Oct 2, 2026
Research Note: Filtering Subversion-Relevant Information From Pretraining Data Is Feasible
LessWrong · Oct 5, 2026