analiza aiqbench
TaskPGM addresses the inefficiency of standard supervised fine-tuning by using an energy-based model to learn optimal task mixtures that balance coverage against redundancy. By modeling task interactions through a Markov random field, the framework provides interpretable structures and approximation guarantees that outperform simple heuristic sampling strategies. This approach offers a principled method for distributing training budgets across heterogeneous tasks to improve transfer performance in large language models.
Learning What Matters: Probabilistic Task Selection via Mutual Information for Model Finetuning
arXiv:2507.12612v3 Announce Type: replace-cross Abstract: Supervised fine-tuning performance for large language models depends strongly on how training budget is distributed across a heterogeneous set of tasks. In practice, mixtures are often fixed using simple heuristics (e.g., uniform or size-proportional sampling) that ignore task interactions, which can hurt transfer and waste budget on redundant sources. We introduce TaskPGM, a framework for learning continuous task mixtures via an energy-based model over tasks. Tasks form the nodes of a Markov random field: unary potentials capture per-task utility, and pairwise potentials encode inter-task relationships using behavioral divergences computed from predictive distributions of single-task fine-tuned models (e.g., Jensen--Shannon divergence and pointwise mutual information). Optimizing this objective yields mixtures that balance coverage against redundancy. We show that the resulting set function is weakly submodular under budget constraints, enabling approximation guarantees for discrete selection variants. Across multiple model families (LLaMA-7B, Qwen2-7B) and evaluation suites (BIG-Bench Hard), TaskPGM improves over standard mixing strategies and provides interpretable structure over task interactions.
Abstract:Supervised fine-tuning performance for large language models depends strongly on how training budget is distributed across a heterogeneous set of tasks. In practice, mixtures are often fixed using simple heuristics (e.g., uniform or size-proportional sampling) that ignore task interactions, which can hurt transfer and waste budget on redundant sources. We introduce TaskPGM, a framework for learning continuous task mixtures via an energy-based model over tasks. Tasks form the nodes of a Markov random field: unary potentials capture per-task utility, and pairwise potentials encode inter-task relationships using behavioral divergences computed from predictive distributions of single-task fine-tuned models (e.g., Jensen--Shannon divergence and pointwise mutual information). Optimizing this objective yields mixtures that balance coverage against redundancy. We show that the resulting set function is weakly submodular under budget constraints, enabling approximation guarantees for discrete selection variants. Across multiple model families (LLaMA-7B, Qwen2-7B) and evaluation suites (BIG-Bench Hard), TaskPGM improves over standard mixing strategies and provides interpretable structure over task interactions.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
I.2.7; I.2.6; I.2.4
arXiv:2507.12612 [cs.LG]
(or arXiv:2507.12612v3 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 174 cuvinte
Subiecte conexe
If This Were a Test, How Much Would It Cost?
LessWrong · 16 iun. 2026
The Entangled Dimensions of Decision Theory
LessWrong · 30 iul. 2026
The Next Swan: Frank Ramsey, Variable Hypotheticals, and the Bet on Induction
LessWrong · 8 iun. 2026
Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
AI Alignment Forum · 3 aug. 2026