aiqbench analysis
This research introduces a rigorous information-theoretic framework for evaluating data curation by leveraging pointwise mutual information, directly addressing the pitfalls of Goodhart's law in benchmark-based assessments. By estimating Shannon mutual information between curated and test datasets via Bayesian models, the study provides a principled metric that penalizes strategies which merely overfit to evaluation benchmarks rather than enhancing true model informativeness. This approach offers a critical advancement for ensuring data quality metrics reflect genuine improvements in model parameters rather than superficial performance gains.
Proper Dataset Valuation by Pointwise Mutual Information
arXiv:2405.18253v4 Announce Type: replace Abstract: Data plays a central role in advancements in modern artificial intelligence, with high-quality data emerging as a key driver of model performance. This has prompted the development of principled and effective data curation methods in recent years. However, existing methods largely rely on heuristics, and whether they are truly effective remains unclear. For instance, standard evaluation methods that assess a trained model's performance on specific benchmarks may incentivize assigning high scores to data that merely resembles the test set. This issue exemplifies Goodhart's law: when a measure becomes a target, it ceases to be a good measure. To address this issue, we propose an information-theoretic framework for evaluating data curation methods. We define dataset quality in terms of its informativeness about the true model parameters, formalized using the Blackwell ordering of informativeness. Under this ordering, Blackwell's theorem ensures that more informative data yields optimal models with lower expected loss on the true underlying distribution. To measure informativeness, we show that the Blackwell order can be determined by the Shannon mutual information between the curated data and the test data. To estimate this mutual information, we introduce a novel method that trains Bayesian models on embedded datasets and computes mutual information from the posteriors of model parameters. Experiments on real-world data demonstrate that our mutual information-based evaluation assigns appropriately lower scores to data curation strategies that reduce dataset informativeness, while traditional test score-based evaluation methods may favor data curation strategies that overfit to the test set but compromise the training data's informativeness.
Abstract:Data plays a central role in advancements in modern artificial intelligence, with high-quality data emerging as a key driver of model performance. This has prompted the development of principled and effective data curation methods in recent years. However, existing methods largely rely on heuristics, and whether they are truly effective remains unclear. For instance, standard evaluation methods that assess a trained model's performance on specific benchmarks may incentivize assigning high scores to data that merely resembles the test set. This issue exemplifies Goodhart's law: when a measure becomes a target, it ceases to be a good measure. To address this issue, we propose an information-theoretic framework for evaluating data curation methods. We define dataset quality in terms of its informativeness about the true model parameters, formalized using the Blackwell ordering of informativeness. Under this ordering, Blackwell's theorem ensures that more informative data yields optimal models with lower expected loss on the true underlying distribution. To measure informativeness, we show that the Blackwell order can be determined by the Shannon mutual information between the curated data and the test data. To estimate this mutual information, we introduce a novel method that trains Bayesian models on embedded datasets and computes mutual information from the posteriors of model parameters. Experiments on real-world data demonstrate that our mutual information-based evaluation assigns appropriately lower scores to data curation strategies that reduce dataset informativeness, while traditional test score-based evaluation methods may favor data curation strategies that overfit to the test set but compromise the training data's informativeness.
Machine Learning (cs.LG); Computer Science and Game Theory (cs.GT)
arXiv:2405.18253 [cs.LG]
(or arXiv:2405.18253v4 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2405.18253 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 349 words
Related coverage
AI-powered metadata correction and harmonization
AWS · Aug 24, 2026
PromptNCE: Conditional Probabilities and PMI Using Only LLMs and Contrastive Estimation Prompts
arXiv · Sep 2, 2026
Sparse Mutual Information Graph Averaging for Improving Random Indexing Embeddings
arXiv · Aug 7, 2026
Generative Representation Learning for Scalable Latent Tokenization
arXiv · Oct 15, 2026