analiza aiqbench
This paper introduces READ, a novel distributionally robust optimization framework that leverages external knowledge about low-dimensional representations to guide worst-case performance estimation. By differentiating transport costs based on representation coordinates rather than treating all feature perturbations equally, the method offers more efficient protection against distributional shifts while maintaining coverage for future populations in transfer learning scenarios. The approach is validated through simulations and a single-cell multi-omics application, demonstrating improved robustness in complex multitask settings where predictive signals are embedded within specific subspaces of covariates.
Representation-Aware Distributionally Robust Optimization: A Knowledge Transfer Framework
arXiv:2509.09371v2 Announce Type: replace-cross Abstract: Distributionally robust optimization (DRO) protects statistical learning against distributional shifts by optimizing the worst-case performance over a set of perturbed distributions. However, standard DRO formulations often treat all feature perturbations equally. This can be unnecessarily conservative when external knowledge suggests that the predictive signal is embedded in a low-dimensional representation of covariates. We propose REpresentation-Aware Distributionally robust estimation (READ), a Wasserstein DRO framework that uses external representations to guide the geometry of robustness. Rather than uniformly perturbing all covariate directions, READ increases the transport cost of perturbations that change representation coordinates, thereby reshaping the dual regularization toward the representation subspace. Meanwhile, it preserves protection against variations orthogonal to the representation. We study READ in two regimes. First, for inference on the current target, we characterize our estimator asymptotically and develop a Wasserstein profile inference approach to construct representation-aligned confidence regions while enabling automatic hyperparameter tuning. Second, for deployment to future populations that differ from the current target but are generated from the same representation-invariant random-coefficient model, we show that the resulting regions achieve higher coverage of future model parameters than standard methods. Simulations and a single-cell multi-omics application demonstrate the advantages of READ in multi-source and multitask transfer learning settings.
Abstract:Distributionally robust optimization (DRO) protects statistical learning against distributional shifts by optimizing the worst-case performance over a set of perturbed distributions. However, standard DRO formulations often treat all feature perturbations equally. This can be unnecessarily conservative when external knowledge suggests that the predictive signal is embedded in a low-dimensional representation of covariates. We propose REpresentation-Aware Distributionally robust estimation (READ), a Wasserstein DRO framework that uses external representations to guide the geometry of robustness. Rather than uniformly perturbing all covariate directions, READ increases the transport cost of perturbations that change representation coordinates, thereby reshaping the dual regularization toward the representation subspace. Meanwhile, it preserves protection against variations orthogonal to the representation. We study READ in two regimes. First, for inference on the current target, we characterize our estimator asymptotically and develop a Wasserstein profile inference approach to construct representation-aligned confidence regions while enabling automatic hyperparameter tuning. Second, for deployment to future populations that differ from the current target but are generated from the same representation-invariant random-coefficient model, we show that the resulting regions achieve higher coverage of future model parameters than standard methods. Simulations and a single-cell multi-omics application demonstrate the advantages of READ in multi-source and multitask transfer learning settings.
Methodology (stat.ME); Machine Learning (cs.LG)
arXiv:2509.09371 [stat.ME]
(or arXiv:2509.09371v2 [stat.ME] for this version)
https://doi.org/10.48550/arXiv.2509.09371 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 294 cuvinte
Subiecte conexe
Continual Learning with Elastic Regularization and Synthetic Replay for Federated MLLM Fine-Tuning
arXiv · 7 oct. 2026
A Red Line and Oversight Framework for Government AI Contracts
AI Alignment Forum · 18 iul. 2026
PIRAMID: Progress and Plans
LessWrong · 27 iul. 2026
BRICS Draws a New Global Line: Full Text of New Delhi Declaration
slguardian.org · 12 sept. 2026