Versatile Incremental Learning : towards class and domain-agnostic incremental learning

다국어 초록 (Multilingual Abstract)

Incremental Learning (IL) aims to accumulate knowledge from sequential input tasks while overcoming catastrophic forgetting. Existing IL methods typically assume that an incoming task has only increments of classes or domains, referred to as Class IL (CIL) or Domain IL (DIL), respectively.

In this work, we consider a more challenging and realistic but under-explored IL scenario, named Versatile Incremental Learning (VIL), in which a model has no prior of which of the classes or domains will increase in the next task. In the proposed VIL scenario, the model faces intra-class domain confusion and inter-domain class confusion, which makes the model fail to accumulate new knowledge without interference with learned knowledge. To address these issues, we propose a simple yet effective IL framework, named Incremental Classifier with Adaptation Shift cONtrol (ICON). Based on shifts of learnable modules, we design a novel regularization method called Cluster-based Adaptation Shift conTrol (CAST) to control the model to avoid confusion with the previously learned knowledge and thereby accumulate the new knowledge more effectively. Moreover, we introduce an Incremental Classifier (IC) which expands its output nodes to address the overwriting issue from different domains corresponding to a single class while maintaining the previous knowledge.

By extending the concept of Versatile Incremental Learning to online incremental learning setting, we introduce Online Versatile Incremental Learning (OVIL) and design a method for the OVIL task.

Continual learning allows models to evolve continuously across diverse, real-world distributions. It has attracted significant attention and driven a range of learning paradigms and evaluation techniques. To better simulate the dynamic and varied visual inputs encountered in real-world exploration, we introduce Online Versertile Incremental Learning (OVIL). In this novel scenario, both classes and domains evolve continuously in an online setting. For the broader scope of changes in input distribution, We analyze model internals to identify hierarchical knowledge encoding structures. This leads to our Consistent Knowledge Guidance (CKG) method, which enables models to prioritize abstract, generalizable knowledge over specific details.

We conducted extensive experiments on widely used benchmarks to demonstrate the effectiveness of our method in all scenarios, especially when the following tasks are randomly changed and, furthermore, in an online setting.

목차 (Table of Contents)

• 1. Introduction 1
• 2. Related Work 5
• 2.1 More Realistic IL Scenario 5
• 2.2 Regularization for IL 5
• 2.3 Model Expansion for IL 6
• 2.4 Online IL 7
• 3. Method 8
• 3.1 Scenario Description of VIL 8
• 3.2 Cluster-based Adaptation Shift Control 9
• 3.3 Incremental Classifier 11
• 3.4 Training Objective 13
• 3.5 Scenario Description of OVIL 13
• 3.6 Anlysis of ViT for distribution changes 14
• 3.7 Geodesic Flow Kernel 16
• 3.8 Consistent Knowledge Guidance 17
• 4. Experiments 19
• 4.1 Experimental Setup 19
• 4.1.1 Datasets 19
• 4.1.2 Comparison Baselines 19
• 4.1.3 Evaluation Metrics 20
• 4.2 Experimental Results 20
• 4.2.1 Main Results 20
• 4.2.2 Results on Cross-Domain Incremental Learning 20
• 4.2.3 Results on OVIL 22
• 4.3 Analysis 22
• 4.3.1 Ablation Study 22
• 4.3.2 Number of Clusters 23
• 5. Conclusion 24
• Supplementary Material 25
• Reference 39