analiza aiqbench
This research introduces MUtE, a framework that unifies concept erasure and counterfactual generation to enhance model interpretability and algorithmic fairness. By deriving novel erasure functions that align with the geometric structure of modern language models, the method effectively removes specific biases while preserving unrelated information. The approach offers a practical bridge between theoretical optimality in representation learning and real-world applications in natural language processing.
MUtE: A Dual Framework for Concept Erasure and Counterfactual Interventions
arXiv:2609.11253v1 Announce Type: new Abstract: Erasing concept-specific information from representations has been proven useful for mitigating bias or interpreting model decisions. The joint objective is to transform the original representations such that the target concept becomes unpredictable, while maximally preserving concept-unrelated information. In this work, we revisit the optimal bounds of concept erasure to derive a novel class of erasure functions that naturally induce a deterministic, dual counterfactual mapping. Bridging the gap between theoretical optimality and practical representation learning, we design an implementation that imposes a translational bias on counterfactual trajectories - a constraint that aligns with how many concepts geometrically manifest in modern language models. Our framework enables seamless navigation between concept erasure and counterfactual generation. We empirically demonstrate its efficacy in improving downstream algorithmic fairness and generating counterfactual texts.
Abstract:Erasing concept-specific information from representations has been proven useful for mitigating bias or interpreting model decisions. The joint objective is to transform the original representations such that the target concept becomes unpredictable, while maximally preserving concept-unrelated information. In this work, we revisit the optimal bounds of concept erasure to derive a novel class of erasure functions that naturally induce a deterministic, dual counterfactual mapping. Bridging the gap between theoretical optimality and practical representation learning, we design an implementation that imposes a translational bias on counterfactual trajectories - a constraint that aligns with how many concepts geometrically manifest in modern language models. Our framework enables seamless navigation between concept erasure and counterfactual generation. We empirically demonstrate its efficacy in improving downstream algorithmic fairness and generating counterfactual texts.
Machine Learning (cs.LG); Computation and Language (cs.CL)
arXiv:2609.11253 [cs.LG]
(or arXiv:2609.11253v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.11253 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 224 cuvinte
Subiecte conexe
CleanVideo: Adaptive Concept Erasure for Text-to-Video Diffusion Models
arXiv · 18 sept. 2026
Against Export Controls (and China Threat Models)
LessWrong · 24 sept. 2026
Beyond Explanation: Debugging Medical Imaging Models via Concept Intervention
arXiv · 8 oct. 2026
EraseSAE: Surgical Concept Erasure in Text-to-Video Diffusion Models via Sparse Autoencoders
arXiv · 5 sept. 2026