analiza aiqbench
MIPO introduces a contrastive data augmentation technique that leverages intrinsic mutual information between prompts and responses to enhance LLM performance without requiring additional labeled data or external verifiers. By constructing preference pairs from the base model's own outputs, this method achieves significant gains in both personalization and verifiable domains like math, addressing the scarcity of new training data. The approach offers a promising path toward self-improvement frameworks that reduce dependency on expensive human supervision.
Maximizing Mutual Information Between Prompt and Response Improves LLM Performance With No Additional Data
arXiv:2603.19294v4 Announce Type: replace-cross Abstract: While post-training has successfully improved large language models (LLMs) across a variety of domains, these gains heavily rely on human-labeled data or external verifiers. Existing data has already been exploited, and new data is expensive to collect. Moreover, true intelligence goes far beyond verifiable tasks. Therefore, we need self-improvement frameworks that are less dependent on external signals and more broadly applicable to both verifiable and non-verifiable domains. We propose **Mutual Information Preference Optimization (MIPO)**, a contrastive data augmentation method that constructs preference pairs by generating a positive response conditioning on the correct prompt, and a negative response by conditioning on a random, unrelated prompt. We show that using Direct Preference Optimization to learn from this paired data maximizes pointwise mutual information *under the base LLM* between prompts and model responses. Experiments with with 1-7B parameter Llama and Qwen instruct models show that MIPO achieves 3-16% gains (and 51% increase for Qwen2.5-1.5B-Instruct) on personalization compared to prompting baselines. Surprisingly, MIPO can also be useful in verifiable domains, such as math and multiple-choice question answering, yielding 1-20% gains *without any additional data or external supervision*. These results suggest a promising direction for self-improvement using intrinsic signals derived from contrastive data pairs.
Abstract:While post-training has successfully improved large language models (LLMs) across a variety of domains, these gains heavily rely on human-labeled data or external verifiers. Existing data has already been exploited, and new data is expensive to collect. Moreover, true intelligence goes far beyond verifiable tasks. Therefore, we need self-improvement frameworks that are less dependent on external signals and more broadly applicable to both verifiable and non-verifiable domains. We propose **Mutual Information Preference Optimization (MIPO)**, a contrastive data augmentation method that constructs preference pairs by generating a positive response conditioning on the correct prompt, and a negative response by conditioning on a random, unrelated prompt. We show that using Direct Preference Optimization to learn from this paired data maximizes pointwise mutual information *under the base LLM* between prompts and model responses. Experiments with with 1-7B parameter Llama and Qwen instruct models show that MIPO achieves 3-16% gains (and 51% increase for Qwen2.5-1.5B-Instruct) on personalization compared to prompting baselines. Surprisingly, MIPO can also be useful in verifiable domains, such as math and multiple-choice question answering, yielding 1-20% gains *without any additional data or external supervision*. These results suggest a promising direction for self-improvement using intrinsic signals derived from contrastive data pairs.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2603.19294 [cs.LG]
(or arXiv:2603.19294v4 [cs.LG] for this version)
Text integral sub licența deschisă a publicației · 217 cuvinte
Subiecte conexe
Reliance Global Group Reports Second Quarter 2026 Results and Accelerates AI - Driven Transformation
manilatimes.net · 30 iul. 2026
Shaft-integrated Force Sensing with Transformer-based Dynamics Compensation for Telesurgery
arXiv · 3 iun. 2026
Stochastic and quantum autonomous information machine: efficiency fluctuations, precision and optimization
arXiv · 27 oct. 2026
On the Relation between Code Quality and Machine Learning Performance: A Large-scale Empirical Study
arXiv · 11 sept. 2026