aiqbench analysis
InTAct targets catastrophic forgetting in domain-incremental learning by constraining updates to the activation regions that earlier domains relied on, rather than freezing parameters or storing past data. Its main claim is that stabilizing neuron function in shared layers can match or beat current prompt-based methods on standard benchmarks, without architecture-specific changes. Readers should check how it performs beyond the reported benchmarks and what compute and memory overhead the activation-region tracking adds.
InTAct: Interval-based Task Activation Consolidation for Continual Learning
arXiv:2511.17439v3 Announce Type: replace Abstract: Continual learning seeks to acquire new knowledge while preserving previously learned representations. Despite the success of prompt-based methods, they remain fragile in domain-incremental learning scenarios, where shifts in the input distribution cause representation drift in shared layers and lead to forgetting. We introduce InTAct, a method that preserves functional behavior in shared representations without freezing parameters or retaining past data. InTAct constrains updates within activation regions associated with prior domains, while allowing flexible adaptation elsewhere, thereby stabilizing neuron functionality rather than directly restricting parameter values. The approach is architecture-agnostic, integrates seamlessly with existing prompt-based frameworks, and matches or outperforms state-of-the-art methods on domain-incremental benchmarks.
Abstract:Continual learning seeks to acquire new knowledge while preserving previously learned representations. Despite the success of prompt-based methods, they remain fragile in domain-incremental learning scenarios, where shifts in the input distribution cause representation drift in shared layers and lead to forgetting. We introduce InTAct, a method that preserves functional behavior in shared representations without freezing parameters or retaining past data. InTAct constrains updates within activation regions associated with prior domains, while allowing flexible adaptation elsewhere, thereby stabilizing neuron functionality rather than directly restricting parameter values. The approach is architecture-agnostic, integrates seamlessly with existing prompt-based frameworks, and matches or outperforms state-of-the-art methods on domain-incremental benchmarks.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2511.17439 [cs.LG]
(or arXiv:2511.17439v3 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2511.17439 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 199 words