The emergence of large pre-trained networks has revolutionized the AI field, unlocking new possibilities and achieving unprecedented performance. However, these models inherit a fundamental limitation from traditional Machine Learning approaches: their strong dependence on the i.i.d. assumption hinders their adaptability to dynamic learning scenarios. We believe the next breakthrough in AI lies in enabling efficient adaptation to evolving environments — such as the real world — where new data and tasks arrive sequentially. This challenge defines the field of Continual Learning (CL), a Machine Learning paradigm focused on developing lifelong learning neural models. One alternative to efficiently adapt these large-scale models is known Parameter-Efficient Fine-Tuning (PEFT). These methods tackle the issue of adapting the model to a particular data or scenario by performing small and efficient modifications, achieving similar performance to full fine-tuning. However, these techniques still lack the ability to adjust the model to multiple tasks continually, as they suffer from the issue of Catastrophic Forgetting. In this survey, we first provide an overview of CL algorithms and PEFT methods before reviewing the state-of-the-art on Parameter-Efficient Continual Fine-Tuning (PECFT). We examine various approaches, discuss evaluation metrics, and explore potential future research directions. Our goal is to highlight the synergy between CL and Parameter-Efficient Fine-Tuning, guide researchers in this field, and pave the way for novel future research directions.

Continual Learning , also known as incremental or lifelong learning [3, 4, 5, 6, 7], is a subfield of machine learning concerned with incrementally acquiring, updating, and retaining knowledge over a sequence of tasks presented over time. Unlike traditional machine learning, which works with static data distributions, continual learning tackles the challenge of learning in a dynamic and evolving environment where new data often come from different distributions. More formally, CL considers a stream of \( T \) tasks. Each task \( t \) comprises a new dataset \( D^t = (X^t,Y^t) \), where \( X^t \) denotes the input instances and \( Y^t \) denotes the instance labels. The objective is to train a model \( f_{\Theta}: X \longrightarrow Y \) using data from a sequence of \( T \) tasks: \( D = \{D^1, ..., D^T \} \), and where each \( D^t\) can follow a different distribution. Here, \( \theta\) are the learned parameters of the model. During each task, the model \( f_{\Theta} \) minimizes the objective function \( \mathcal{L} \) using data \( D^t \). Each task is presented sequentially to the model and trained for \( E\) epochs. The objective function is defined as follows:

2.1.1 CL Challenges

2.1.2 CL Scenarios

Large Pre-trained Models are massive neural networks trained on vast amounts of data to learn general-purpose representations [19, 20]. These models, predominantly based on the Transformer architectures [21], have significantly altered the machine learning landscape, particularly in natural language processing and increasingly in computer vision and multimodal applications.

The “pre-training” in PTMs refers to the initial training phase where the model learns to perform tasks on large corpora, such as predicting masked words or generating coherent text. This process imbues the model with a broad understanding of language structures, world knowledge, and even rudimentary reasoning capabilities [22]. The “large” aspect denotes the model’s size in terms of parameters and the scale of data and computation involved in their training [23].

A key advantage of PTMs is their capacity to learn robust representations that serve as excellent starting points for a wide range of downstream tasks, often requiring minimal task-specific training data [24]. In practice, pre-trained models are adapted to specific tasks through fine-tuning or by training new classifier layers, making them versatile tools across various domains.

Key examples of large PTMs showcase their diverse capabilities:

• BERT [19] revolutionized natural language understanding tasks.
• GPT-3 [20] demonstrated remarkable text generation abilities.
• T5 [24] unified various NLP tasks under a single model.
• Vision Transformers [25] applied similar principles to image recognition.

These models have pushed the boundaries of artificial intelligence, exhibiting capabilities that range from human-like text generation to solving complex reasoning tasks, often with little to no task-specific training [26]. Their effectiveness in capturing knowledge from large volumes of labeled and unlabeled data, combined with their adaptability, has made PTMs crucial in multiple research and application areas.

The fundamental architecture commonly used with large PTMs is the Transformer [21], built around the self-attention mechanism. The way this layer works is that given an input sequence \( X = \{x_1, x_2, …, x_n\} \), where \( x_i \) represents the \(i\)-th token or patch, the Transformer module computes attention scores using queries \( Q \), keys \( K \), and values \( V \), which are linear projections of \( X \). Self-attention mechanism is defined as:

where \( d_k \) is the dimensionality of the keys. This attention allows the model to weigh the importance of each token/patch relative to others, capturing contextual relationships efficiently. This architectural component makes language models excel in natural language understanding and generation by employing multiple layers of self-attention and feed-forward networks. These models are trained to generate text by predicting the next token or finding the masked value in a sequence, maximizing the probability \( P(x_{i+1} | x_1, x_2, …, x_i) \).

There exists also vision models, e.g. ViT [25], that apply the Transformer architecture to image data. An image is divided into patches, flattened and linearly embedded into a sequence of vectors. Given an image \( I \) divided into \( N \) patches, each patch \( p_i \) is embedded into a vector \( e_i \). The sequence of embedded patches \( E = \{e_1, e_2, …, e_N\} \) is then processed through the transformer architecture similarly to text tokens, allowing the model to capture spatial relationships within the image.

Although PTMs have demonstrated remarkable success across various tasks, they have some limitations. One significant challenge is the domain gap between pre-training and target task data. Pre-trained models are typically trained on large-scale datasets; however, in many cases, these datasets will not fully represent the specific characteristics of the target task. This domain gap can lead to performance degradation when deploying pre-trained models directly to new tasks or domains [27].

One way of alleviating this problem is to employ Transfer Learning strategies [28]. Transfer learning involves leveraging knowledge from a source domain to improve performance on a target domain or task. By fine-tuning the model parameters on a smaller dataset related to the target task, the model can adapt to the specific characteristics of the task while using the pertaining as a good starting point. Parameter-Efficient Fine-Tuning has emerged as a significant advancement in large PTMs studies. It refines the capabilities of pre-trained models by strategically adjusting a limited subset of parameters during fine-tuning. This approach differs significantly from comprehensive fine-tuning, which requires the entire model to be retrained, often at considerable computational cost. PEFT achieves impressive results while minimizing parameter updates, ensuring efficient adaptation and preserving the model’s previously acquired knowledge [29].

One alternative to naively training a model in a sequence of tasks is adding a constraint on the loss functions to encourage the mitigation of forgetting. This regularization can be imposed on weights by estimating the parameters importance so relevant weights do not drift significantly. One of the first approaches that proposed this was Elastic Weight Consolidation (EWC) [32], which captures the prior importance using a diagonally approximated Fisher information matrix. EWC is later improved by finding a better approximation of the fisher information matrix (sFIM).

Regularization can also be imposed on activations to prevent activation drift, which generally outperforms its weight-regularisation counterpart. One work in this line is Learning without Forgetting (LwF) [33], which prevents activations of the old network from drifting while learning new tasks. The less-forgetting learning penalizes the activation difference except for the fully-connected layer. Riemannian Walk [34] extends EWC by incorporating KL-divergence-based regularization, path integral methods to assess parameter importance and sample-based strategies for retaining past knowledge. Rotated EWC (R-EWC) [35] enhances EWC by reparameterizing the network to better align with the FIM, improving the diagonal approximation and reducing forgetting.

Similarly, Synaptic Intelligence (SI) [36] evaluates the importance of each synapse online during training and consolidates significant parameters by penalizing considerable changes to mitigate forgetting. Memory Aware Synapses (MAS) [37] determine the importance of weights based on the sensitivity of the learned function, enabling adaptive penalization of changes to significant weights without relying solely on the loss function. Efforts have also been directed towards enhancing the implementation of secondary penalties. Incremental Moment Matching (IMM) [38] integrates multiple models trained on different tasks by aligning their weight distributions, employing weight transfer, and L2 regularization to sustain performance across tasks.

Function regularization is another way, also known as Knowledge Distillation (KD) [39], to target the intermediate or final output of the prediction function. This approach typically employs the previously-learned model as the teacher and the currently-trained model as the student, leveraging knowledge distillation techniques to mitigate catastrophic forgetting. In ideal scenarios, KD would target all old training samples, but in continual learning settings, alternatives such as new training samples, a small fraction of old training samples, external unlabeled data, or generated data are used, albeit suffering from varying degrees of distribution shift.

Pioneering works like iCaRL [40] learn new training samples while utilizing predictions from the output heads of the old tasks to compute the distillation loss. LwM [41] exploits the attention maps of new training samples for KD, while EBLL [7] learns task-specific autoencoders to prevent changes in feature reconstruction. GD [42] further distills knowledge from the large stream of unlabeled data available.

Some works add three subgroups [30]: Logit Distillation, Feature Distillation, and Relational Distillation. Logit distillation focuses on transferring knowledge by aligning the final output logits of the neural network between the old and new models. LwF [33] is a pioneering application of KD in CL, guiding the learning of the new model by utilizing the outputs of the old model.

Feature distillation ensures the protection of learned knowledge by distilling at the intermediate levels of the models. Methods like UCIR [43] force the features extracted by the new embedding module to be the same as the old one, providing a stronger regularization. Other works like LwM [44] and AFC [45] utilize attention maps and feature importance, respectively.

Relational distillation introduces a different perspective by distilling the relationships between multiple samples or instances. This level captures the structural information within the data, enhancing the retention of old knowledge. Methods like COIL [46] suggest bidirectional distillation with co-transport, utilizing semantic relationships between old and new models.

Replay-Based methods stores a small subset of data from the previously accessed tasks to reinforce the network’s memory of old knowledge. Current memory-based methods have achieved promising results on many CL benchmarks. Saving examples helps mitigate forgetting from previous tasks by representing past distributions used during training. For the memory to be sufficient, it must represent the previous distribution as fully as possible, considering all its classes and concepts.

For the case of replay-based methods, together with minimizing Equation 1, model \( f_{\Theta}\) needs to minimize \( \mathcal{L}\) using the data available in memory \( M\) at time \( t\) . The buffer \( M^t\) comprises \( |M|\) samples from previous distributions, meaning that at task \( t\) , the buffer will contain samples only from \( t'<t\) . As shown in Equation 4.

The function in charge of populating memory \( M\) is known as Storage Policy [47] and decides which elements go into the memory by sampling from set \( D^t\) given a function \( \mathbf{P}\) , as shown in Equation 5. An ideal policy function is the one that minimizes Equation 4 for evaluation stream \( D^1 ... D^T\) , restricted by the memory size \( |M|\) .

In most cases, we assume that \( M^{t+1}\) will always contain \( |M|\) samples, and the storage policy will decide which samples to remove to add those from new task \( D^t\) . In practice, designing an effective storage policy involves striking a balance between preserving diversity in the memory, i.e. representing past distributions fully, and accommodating new information from the current task. This delicate balance ensures that the memory remains relevant and informative over time, contributing to the model’s overall performance across the entire sequence of tasks. The replay-based CIL branch quickly draws attention due to its appealing ability to resist catastrophic forgetting. For instance, PODNet [48] adopts an efficient spatial-based distillation loss to reduce forgetting, with a focus on the large-phase setting, achieving reasonably good results. AANets [49] employs a new architecture containing a stable block and a plastic block to balance the stability and plasticity.
On top of the replay-based CIL, methods exploring exemplar storing techniques are also fruitful. For instance, Reinforced Memory Management (RMM) [50] seeks dynamic memory management using reinforcement learning. By plugging it into PODNet and AANets, RMM attains a state-of-the-art performance.
GAN-based CIL replays past samples by generating them using GANs. The deep generative replay generates synthetic samples using an unconditioned GAN. It is later improved by memory replay GAN adopting a label-conditional GAN. GAN-based CIL relies heavily on GAN’s generative performance and is only tested on relatively small datasets, such as MNIST. Bias correction-based CIL mainly tries to address the task-recency bias. The end-to-end incremental learning reduces the bias by introducing a balance training stage where only an equal number of samples for each class is used. The Bias Correction (BiC) [51] includes an additional trainable layer which aims to correct the bias. The method named LUCIR [52] fights the bias by changing the softmax layer into a cosine normalization one.

In architecture-based methods of CL, the network architecture is dynamically updated during the learning process to retain previously acquired knowledge. These methods are designed to adjust the model’s architecture to effectively handle new tasks while maintaining the knowledge from earlier tasks. Architecture-based methods can be categorized into fixed-capacity and capacity-increasing approaches based on the evolution of their model parameters as the number of tasks increases. Building on these foundations, we explore the implementation of task-specific parameters, extending these concepts through parameter allocation and dynamic network architectures.

Parameter Allocation approach dedicates isolated parameter subspaces within the network to each task. Piggyback [53], HAT [54], WSN [55], and H2 [56] utilize binary masks to select specific neurons or parameters for each task, effectively freezing the old tasks’ masked regions to prevent forgetting. PackNet [57], CLNP [58], and AGS-CL [59] identify important neurons or parameters for the current task and release the rest for subsequent tasks. However, limited network capacity can lead to saturation of “free” parameters as the number of tasks increases. Dynamic architecture expansion can address this by allowing the network to grow when necessary, with methods like reinforcement learning, architecture search, and variational Bayes being employed for optimization.

Dynamic network architecture could classify model decomposition and modular network. Model Decomposition explicitly separates a model into task-sharing and task-specific components. Regular networks can incorporate task-specific components as parallel branches, adaptive layers, or masks of intermediate features. Additionally, network parameters themselves can be decomposed into task-sharing and task-specific elements. While this approach offers scalability, the number of task-specific components typically grows linearly with tasks, making resource efficiency a crucial factor.

Modular Network utilize parallel sub-networks or sub-modules to learn tasks independently. Early works like Progressive Networks [60] introduced an identical sub-network for each task, facilitating knowledge transfer through adapter connections. Expert Gate [4] employs a mixture of experts, expanding one for each new task. PathNet [61] pre-allocates parallel networks to construct candidate paths, selecting the best path for each task. MNTDP [62] seeks to find the optimal layout from existing and potentially new sub-modules. Similar to Parameter Allocation, these methods aim to construct task-specific models while enabling explicit knowledge reuse through sub-network combinations.

Unlike other directions, most architecture-based approaches aim to de-correlate tasks in network parameters, potentially sacrificing scalability and inter-task generalizability. Task identities are often required to determine which set of parameters to use. This limitation can be overcome by inferring task identities from the predictive uncertainty of task-specific models or by learning the function of task-identity prediction through continual learning strategies.

Beyond incorporating additional loss terms, e.g. regularization and replay, the optimization-based approach in CL explores alternative optimization strategies. This includes techniques like gradient projection to constrain model updates and prevent knowledge loss, as well as meta-learning approaches that aim to automatically acquire inductive biases suitable for CL scenarios. Additionally, some work focuses on optimizing the training process from a loss landscape perspective.

CL encompasses a diverse range of techniques that go beyond regularization and experience replay to address the challenge of learning from a continuous stream of data without forgetting previously acquired knowledge. Gradient projection methods, such as GEM [63], A-GEM [64], and LOGD [65], manipulate parameter updates to align with or remain orthogonal to specific directions, preserving past input and gradient spaces. In contrast, OWM [66] and OGD [67] offer contrasting strategies: OWM modifies updates to be orthogonal to the previous input space, while OGD preserves old gradient directions and rectifies current ones. OrthogSubspace [68] and GPM [69] leverage orthogonal subspaces for CL, with [69] focusing on maintaining the gradient subspace of old tasks. FS-DGPM [70] dynamically adjusts [69] by discarding unimportant bases to enhance learning plasticity and convergence. TRGP [71] defines a “trust region” based on gradient projection to selectively reuse frozen weights from prior tasks.

Meta-learning approaches in CL aim to acquire a data-driven inductive bias suitable for various scenarios, eliminating the need for manual design. OML [72] employs meta-training to perform online updates while minimizing interference, promoting sparse representations. ANML [73] extends this idea by meta-learning a context-dependent gating function to activate relevant neurons for each task. Meta-learning can be combined with experience replay for better utilization of both old and new data. MER [74] aligns gradient directions, while iTAML [75] applies a meta-updating rule for balancing them. La-MAML [76] optimizes the OML objective with an adaptive learning rate and leverages replay for online training. OSAKA [77] proposes a hybrid objective for knowledge accumulation and fast adaptation, achieved through meta-training initialization and task incorporation. MERLIN [78] utilizes a metadistribution of model parameters to sample and ensemble task-specific models for inference, while MARK [79] maintains incrementally updated shared weights through meta-learning and selective masking for specific tasks.

In additive methods, task-specific parameters and lightweight modules are added to the pre-trained model architecture.

4.1.1 Adapter Based PEFT Methods

4.1.2 Prompt Based PEFT Methods

Reparameterization methods provide an alternative way to represent and update a neural network’s parameters during training. Instead of directly modifying the original weight matrices, it expresses them through a transformation function using a smaller set of parameters. The key advantage is that we can achieve similar model capabilities while training far fewer parameters, making the process more efficient and controllable.

LoRA [91] addresses the challenge of adapting massive pre-trained models to specific tasks by using low-rank decomposition matrices to indirectly update the model’s weights. It introduces a low-rank update to the pre-trained model weights, expressed as:

where \( W_0 \in \mathbb{R}^{d \times k}\) represents the initial pre-trained weight matrix, while \( B \in \mathbb{R}^{d \times r}\) and \( A \in \mathbb{R}^{r \times k}\) are the additional parameters, with rank \( r \ll \min(d,k)\) . During finetuning, \( W_0\) is kept frozen while the \( A\) and \( B\) matrices are updated. This formulation allows for efficient computation and storage, as the update matrices \( B\) and \( A\) are much smaller than the original weight matrix. LoRA is particularly efficient both for the low parameter count and for the small GPU usage during training. Furthermore, it introduces no additional latency during inference, as the update \( \Delta W = BA\) can be merged with \( W_0\) post-training.

QLoRA [96] is a reparameterization approach that enables training of LLMs on limited hardware while maintaining performance. It achieves this by combining quantization with [91] through several key innovations. QLoRA processes weight matrices through a dual approach:

where the base model weights are quantized to 4-bit precision using NormalFloat (NF4), while the LoRA adapters (\( L_1, L_2\) ) operate in 16-bit precision. QLoRA integrates three major things that work together to enable efficient training: 4-bit NormalFloat (NF4), an information-theoretically optimal quantization format for normally distributed weights that outperforms standard 4-bit quantization; Double Quantization, which further compresses the model by quantizing the quantization constants themselves, reducing memory requirements by approximately 0.37 bits per parameter; and Paged Optimizers that manage memory spikes during training by utilizing NVIDIA unified memory for automatic CPU-GPU memory transfers. This combination enables fine-tuning of 65B parameter models on a single 48GB GPU while preserving full 16-bit performance. In practice, QLoRA achieves comparable results to full fine-tuning while requiring only 0.1% of the trainable parameters, demonstrating particular effectiveness in low-data regimes and better extrapolation to unseen topics.

DoRA [97] also uses low-rank matrix decomposition, but its novelty lies in decomposing the pre-trained weight matrix \( W\) into magnitude \( m\) and direction \( V\) components:

where \( \|\cdot\|_c\) denotes the vector-wise norm across each column.
It then applies LoRA specifically to the directional component:

where \( W_0\) is the pre-trained weight, and \( BA\) represents the low-rank update. This decomposition approach is inspired by Weight Normalization techniques and aims to simplify the learning task for the low-rank updates. By doing so, DoRA seeks to enhance the learning capacity and stability of LoRA, potentially bridging the performance gap between PEFT methods and full fine-tuning. Importantly, [97] maintains the inference efficiency of LoRA, introducing no additional latency during deployment.

On the other hand, RoSA (Robust Adaptation) [98] combines both low-rank and sparse adapters trained in parallel with frozen pretrained weights. Inspired by robust principal component analysis, RoSA recognizes that while fine-tuning updates can be approximated by low-rank matrices [91], adding a sparse component can capture important outlier components that LoRA might miss. The method achieves comparable or better accuracy than full fine-tuning while maintaining the memory and computational efficiency of other PEFT methods, making it particularly effective for complex tasks like mathematical reasoning and coding where traditional LoRA methods often fall short. Additionally, RoSA is compatible with quantized base weights, making it the first approach to successfully combine quantization, low-rank projections, and sparsity while maintaining high performance.

VeRA [99] is another reparameterization PEFT method for large language models that significantly reduces the number of trainable parameters compared to LoRA while maintaining comparable performance. It uses a single pair of shared random matrices across all layers and learns small scaling vectors for adaptation. Unlike LoRA, VeRA uses frozen random matrices \( A\) and \( B\) shared across layers, with trainable scaling vectors \( b\) and \( d\) :

where \( \Lambda_b\) and \( \Lambda_d\) are diagonal matrices formed from vectors \( b\) and \( d\) . Here, \( B \in \mathbb{R}^{m \times r}\) and \( A \in \mathbb{R}^{r \times n}\) are not required to be low-rank since they remain static. The method employs two initialization strategies:

• Shared Matrices: The frozen matrices \( A\) and \( B\) use Kaiming initialization to maintain consistent variance across all ranks
• Scaling Vectors: Vector \( b\) is initialized to zeros, while vector \( d\) is initialized with a single non-zero value across all elements

The family of selective methods for PEFT includes such techniques that aim to pick and finetune only a small subset of the model parameters, hence reducing computational requirements and memory footprint.

Masking [84] starts from the idea of selecting the weights that are important for the downstream task, instead of finetuning the entire model. Based on the lottery ticket hypothesis [100], they propose to select the parameters with a series of learned binary masks, one for each task. The authors experimented with Transformer-based architectures, such as BERT [19], RoBERTa [101] and DistilBERT [102], on different NLP tasks, including part-of-speech tagging, named-entity recognition, sequence classification, and reading comprehension. In this approach, for each weight matrix \( \bf{W}^l\) of the \( l\) -th transformer block, they randomly initialize a matrix \( \bf{M}^l\) , which is then binarized via an element-wise thresholding function. The binary mask is then applied to the weights by Hadamard product:

Through their experiments, they prove that this masking technique achieves comparable results to complete finetuning, while being more parameter-efficient and lightweight. They also show that 3% \( \sim\) 10% initial sparsity of \( \bf{M}^l_\text{bin}\) represents a good trade-off between retaining knowledge and flexibility. Additionally, they demonstrate that, in general, it is reasonable to select all the weights from shallower layers, while learning masks for higher layers to reach optimal results.

BitFit [103] is a sparse finetuning method, in which the basic concept is to update only the bias terms, while keeping the rest of the network frozen. Such approach is very parameter-efficient, since very few weights are modified: for BERT architecture, they amount to less than 0.1% of the total number of the parameters. In particular, the authors consider the bias terms of key, query and value weights of each self-attention head, for each layer of the BERT encoder, plus the bias parameters from the MLP layers on top.
The BitFit approach was evaluated against the GLUE benchmark [104], obtaining comparable results to naive finetuning, but modifying a tiny portion of the weights. Moreover, they experimented with updating a subset of the bias parameters, achieving only slightly worse results.

In recent times, several works in CL [2, 1, 108, 109] have incorporated different PEFT methods, each offering a different complementary advantage over its CL counterpart. In this survey, we split these PECFT method based on the kind of PEFT method applied.

5.1.1 Adapter-based PECFT

5.1.2 LoRA-Based PECFT

5.1.3 Prompt-based PECFT

The problem of correctly evaluating the performance of CL algorithms has been present since the early developments of the field. It is clear that accuracy alone is not enough: most of the times, standard “offline” learning models have an higher accuracy than CL solutions, but this drop may be acceptable if we measure other, but nonetheless equally important benefits brought by CL algorithms.
This is particularly evident in the context of PECFT methods. Indeed, we mentioned multiple times throughout this work that the principal concern about PTMs lies in their training and inference inefficiency, and in the large resources needed for fine-tuning. This means that, when dealing with very large architectures, pure accuracy cannot be the only metrics employed to evaluate such models.

Firstly, in dynamic scenarios the accuracy on the last task only is not so informative, since we are interested in retaining knowledge and performance on all seen task. Formally, we define the Average Accuracy [5] as:

where \( a_{i,j}\) is the accuracy on task \( j\) , with the model trained continually from task 1 to task \( N\) .

Apart from pure accuracy, the CL fields principally studies approaches to diminish the catastrophic forgetting phenomenon, hence it comes straightforwardly that it represents an important measure to keep into consideration.
Average Forgetting [34] represents how much the model “forgets” about previous tasks. It is formally defined as:

Given the sequential nature of the learning scenarios, it is useful to measure what is the influence of learning multiple tasks, i.e. how much training on a task affects the performance on other tasks. This is particularly interesting because it gives practical insights on how fast the model learns through time. For this reasons, forward transfer and backward transfer [131, 132] are introduced.
Forward Transfer(FWT) measures the influence a task \( t\) has when training on a future task \( k\) ; a positive FWT indicates that the model is able to exploit previously acquired knowledge during the subsequent learning sessions. It is defined as:

Similarly, Backward Transfer(BWT) indicates the influence of a task \( t\) on a previously learned task \( k\) ; a positive BWT means the agent is able to not degrade and improve performance throughout its lifetime.

Other meaningful metrics to keep track of are the ones related to the model efficiency. An absolute measure of the model size is given by the Number of trainable parameters [91], i.e. the amount of parameters introduced by a PECFT method. This represents a straightforward yet effective measure of the computational requirements of a given approach, and a simple way to compare different techniques.

On the same line, the relative growth of the number of parameters over time is equally important. We define the Model Size Efficiency (MS) [132] as:

where we indicate with \( Mem(\theta_i)\) the model memory size in terms of the parameter \( \theta\) at the task \( i\) .

Different pre-trained models’ influence on CL methods’ effectiveness requires deeper analysis. Research not only indicates variations in performance between models trained from scratch and those utilizing pre-trained architectures but also shows that different pre-trained models benefit differently from existing continual learning approaches. [133] Thus, exploring the selection of the optimal pre-trained model and its best-fitted continual learning strategy given a real-world task can lead to meaningful contributions to guiding industry practice. Additionally, while a majority of the experiments were tested over visual and natural language modalities, video and audio are domains where the application of continual learning can be valuable to explore as well. [134, 135]

A multimodality pre-trained model poses a challenge but also offers opportunities for developing cross-modality strategies for continual learning tasks. The key idea is that as the models receive additional information from the task, they can develop a more accurate and robust representation, especially when the supplementary guidance comes from large pre-trained models, which are known for their ability to generate robust representations in the first place. For instance, [136] indicates that aligning visual features with semantic groups and leveraging semantic relations among categories can boost model robustness against distribution shifts. While language guidance has been extensively investigated in Transfer Learning [137] across different vision tasks, its utilization in continual learning has been relatively overlooked. With the rise of large pre-trained models in specific domains, exploring potential cross-modality guidance holds promise for further research.

Model merging presents an exciting opportunity in CL, by combining multiple expert models, each specialized in different aspects of a task, we can create a system that not only mitigates issues like catastrophic forgetting but also benefits from the diverse strengths of each model. This becomes particularly important in dynamic, evolving domains, as it allows the model to expand its knowledge over time without forgetting what it has previously learned. However, a common problem that these model merging solutions, such as Task Arithmetic [138], often encounter is parameter interference, which leads to significant performance degradation when these expert models are merged. Some works such as TIES-MERGING [139] and DARE [140] have led to significant improvements in model merging. [139] addresses interference by resetting parameters that have only changed minimally, resolving sign conflicts, and merging only those parameters that align with the final agreed-upon sign. [140], on the other hand, eliminates redundant delta parameters by randomly dropping them and rescaling the remaining ones, which has shown tremendous effectiveness in sparsifying and merging multiple expert models without significant performance loss.

Typical model merging scenarios often require combining pre-existing expert models—each specialized in a specific task—into one unified system. However, this static approach falls short in scenarios where new tasks emerge over time. In continual learning, we face the challenge of incrementally integrating new task-specific models without retraining the entire system. Recent advances in dynamic model merging address this by tackling issues such as parameter interference, memory efficiency, and sequential integration, enabling systems that adapt more effectively as new tasks are encountered. For instance, MagMax[141] introduces a framework that merges task-specific models using sequential fine-tuning combined with a maximum magnitude weight selection strategy. This approach integrates new information effectively and preserves the integrity of earlier learning to help tackle CT. In contrast, Representation Surgery for Multitask Model Learning [142] addresses a different challenge. Here, the focus is on mitigating the representation bias that emerges when merging models trained on disparate tasks. By inserting a lightweight, task-specific module—dubbed “Surgery”—the method realigns the merged model’s internal representations with those of the individual models, thereby enhancing overall performance in multitask scenarios. While Adaptive LoRA Merging for Domain Incremental Learning[143] highlights the limitations of fixed-weight merging by proposing an adaptive mechanism that dynamically computes merging coefficients. This flexibility allows the system to balance the contributions of new and old domains, ensuring robust performance in evolving environments while reducing manual tuning.

Lastly, [144] takes a sequential projection-based approach. By projecting new updates onto subspaces orthogonal to those of previously merged models and applying adaptive scaling, this method minimizes interference and maintains a constant memory footprint, making it highly scalable for continual learning applications.

By moving away from traditional continual learning approaches, recent trends focus on the adaptive combination of lightweight modules such as [145, 91] in dynamic environments. This enables the seamless integration of new tasks as they emerge, without the need for extensive retraining of large, monolithic models. By merging these modular components on the fly, systems can remain both efficient and realistic in handling real-world challenges, making them ideally suited for large-scale models

Large models have been considered to exhibit “emergent” phenomena, yet they still have ways to go before they can effectively handle more complex reasoning tasks. [26, 146] CL is crucial in this regard and will also face new challenges.

The evolution of tasks from single-ability tasks, such as image classification, to more complex reasoning tasks demands a re-evaluation of existing continual learning methods or the proposal of new approaches. Classically, many methods in CL are evaluated using classification datasets such as CIFAR and ImageNet. There has been a recent push towards transitioning from hard split datasets to benchmarks that reflect more natural temporal and task shifts. [147, 148] Despite this, very few studies address the specific requirement for more complex reasoning tasks, such as visual question answering [149, 150, 151], decision-making [152], motion programming [153], scene understanding [154].

Reasoning tasks can differ significantly in their requirements. For instance, in image classification, a model continually improves its ability to recognize objects within a single modality and primarily focuses on learning representations for each class. However, in reasoning tasks like visual question answering, a model in many cases need to acquire multiple new abilities such as attribute recognition, knowledge reasoning based on changing demands. [151, 155]

Representation methods that focus on retain representative embedding from historical task typically fail to handle this situation, thus cannot be applied directly. Traditional replay techniques may prove insufficient in addressing these more intricate tasks as well. Some good examples that dive deeper into the specific need for reasoning tasks are: [156] maps sub-symbolic inputs to high-level concepts and explicitly replay concepts for CL in neuro-symbolic(NeSy) architectures such as DeepProbLog [157];  [151] utilizes scene graph as a prompt to represent previous images in visual question answering task and replays pseudo scene graphs alongside corresponding QA pairs.

Therefore, a promising direction for future research efforts involves developing novel approaches to improve the reasoning ability of large models within the framework of continual learning.

Future research should explore Continual Learning in more practical and realistic settings. This includes scenarios characterized by limited computational budgets. A recent survey [158] pointed out that existing continual learning methods tend to consider memory constraint, but computational costs are not extensively considered. While the cost of memory and privacy concerns may seem prohibitive, in practice, storing large datasets for extended periods can relatively inexpensive compared to the computational costs of training models on such datasets. [159] This is especially relevant for methods relying on replay mechanisms.

Additionally, investigating Online Learning [160, 15] paradigms where data arrives in small, incremental batches presents a significant frontier for research in this field. Inherently, the ability to detect incoming data shifts enables a more efficient model update process and can contribute to more robust model performance.

Task-agnostic [161] and unsupervised [162, 163] are settings were less explored but have started to gain more attention in recent years. Classic Continual Learning settings often assume availability of data annotation such as class label or task label. However, data labeling efforts can be costly and even not feasible in some real-world industrial situations. Thus, research into unsupervised or Task-agnostic settings can lead to more practical use cases of Continual Learning in real-world scenarios.

[1] Zifeng Wang and Zizhao Zhang and Chen-Yu Lee and Han Zhang and Ruoxi Sun and Xiaoqi Ren and Guolong Su and Vincent Perot and Jennifer Dy and Tomas Pfister Learning to prompt for continual learning Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2022 139–149

[2] Zifeng Wang and Zizhao Zhang and Sayna Ebrahimi and Ruoxi Sun and Han Zhang and Chen-Yu Lee and Xiaoqi Ren and Guolong Su and Vincent Perot and Jennifer Dy and others Dualprompt: Complementary prompting for rehearsal-free continual learning European Conference on Computer Vision 2022 631–648 Springer

[3] Zhiyuan Chen and Bing Liu Lifelong machine learning Springer 2018 1

[4] Rahaf Aljundi and Punarjay Chakravarty and Tinne Tuytelaars Expert gate: Lifelong learning with a network of experts Proceedings of the IEEE conference on computer vision and pattern recognition 2017 3366–3375

[5] Arslan Chaudhry and Marc'Aurelio Ranzato and Marcus Rohrbach and Mohamed Elhoseiny Efficient lifelong learning with a-gem arXiv preprint arXiv:1812.00420 2018

[6] German I Parisi and Ronald Kemker and Jose L Part and Christopher Kanan and Stefan Wermter Continual lifelong learning with neural networks: A review Neural networks 2019 113 54–71

[7] Amal Rannen and Rahaf Aljundi and Matthew B Blaschko and Tinne Tuytelaars Encoder based lifelong learning Proceedings of the IEEE international conference on computer vision 2017 1320–1328

[8] Michael McCloskey and Neal J Cohen Catastrophic interference in connectionist networks: The sequential learning problem Psychology of learning and motivation Elsevier 1989 24 109–165

[9] Lucas Caccia and Rahaf Aljundi and Nader Asadi and Tinne Tuytelaars and Joelle Pineau and Eugene Belilovsky New Insights on Reducing Abrupt Representation Change in Online Continual Learning 2022

[10] Julio Hurtado and Dario Salvati and Rudy Semola and Mattia Bosio and Vincenzo Lomonaco Continual learning for predictive maintenance: Overview and challenges Intelligent Systems with Applications 2023 200251

[11] Dongwan Kim and Bohyung Han On the Stability-Plasticity Dilemma of Class-Incremental Learning 2023

[12] Gido M Van de Ven and Andreas S Tolias Three scenarios for continual learning arXiv preprint arXiv:1904.07734 2019

[13] Haizhou Shi and Hao Wang A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm 2023

[14] Liyuan Wang and Xingxing Zhang and Hang Su and Jun Zhu A comprehensive survey of continual learning: Theory, method and application IEEE Transactions on Pattern Analysis and Machine Intelligence 2024

[15] Albin Soutif-Cormerais and Antonio Carta and Andrea Cossu and Julio Hurtado and Vincenzo Lomonaco and Joost Van de Weijer and Hamed Hemati A comprehensive empirical evaluation on online continual learning Proceedings of the IEEE/CVF International Conference on Computer Vision 2023 3518–3528

[16] Hamed Hemati and Andrea Cossu and Antonio Carta and Julio Hurtado and Lorenzo Pellegrini and Davide Bacciu and Vincenzo Lomonaco and Damian Borth Class-Incremental Learning with Repetition 2023

[17] Jihwan Bang and Heesu Kim and YoungJoon Yoo and Jung-Woo Ha and Jonghyun Choi Rainbow Memory: Continual Learning with a Memory of Diverse Samples 2021

[18] Sangdoo Yun and Dongyoon Han and Seong Joon Oh and Sanghyuk Chun and Junsuk Choe and Youngjoon Yoo Cutmix: Regularization strategy to train strong classifiers with localizable features Proceedings of the IEEE/CVF international conference on computer vision 2019 6023–6032

[19] Jacob Devlin and Ming-Wei Chang and Kenton Lee and Kristina Toutanova BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) 2019 4171–4186

[20] Tom Brown and Benjamin Mann and Nick Ryder and Melanie Subbiah and Jared D Kaplan and Prafulla Dhariwal and Arvind Neelakantan and Pranav Shyam and Girish Sastry and Amanda Askell and others Language models are few-shot learners Advances in neural information processing systems 2020 33 1877–1901

[21] Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan N. Gomez and Lukasz Kaiser and Illia Polosukhin Attention Is All You Need 2023

[22] Fabio Petroni and Tim Rocktäschel and Patrick Lewis and Anton Bakhtin and Yuxiang Wu and Alexander H. Miller and Sebastian Riedel Language Models as Knowledge Bases? 2019

[23] Jared Kaplan and Sam McCandlish and Tom Henighan and Tom B. Brown and Benjamin Chess and Rewon Child and Scott Gray and Alec Radford and Jeffrey Wu and Dario Amodei Scaling Laws for Neural Language Models 2020

[24] Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer 2023

[25] Alexey Dosovitskiy and Lucas Beyer and Alexander Kolesnikov and Dirk Weissenborn and Xiaohua Zhai and Thomas Unterthiner and Mostafa Dehghani and Matthias Minderer and Georg Heigold and Sylvain Gelly and Jakob Uszkoreit and Neil Houlsby An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale 2021

[26] Jason Wei and Yi Tay and Rishi Bommasani and Colin Raffel and Barret Zoph and Sebastian Borgeaud and Dani Yogatama and Maarten Bosma and Denny Zhou and Donald Metzler and others Emergent abilities of large language models arXiv preprint arXiv:2206.07682 2022

[27] Xu Guo and Han Yu On the Domain Adaptation and Generalization of Pretrained Language Models: A Survey 2022

[28] Fuzhen Zhuang and Zhiyuan Qi and Keyu Duan and Dongbo Xi and Yongchun Zhu and Hengshu Zhu and Hui Xiong and Qing He A Comprehensive Survey on Transfer Learning 2020

[29] Zihao Fu and Haoran Yang and Anthony Man-Cho So and Wai Lam and Lidong Bing and Nigel Collier On the effectiveness of parameter-efficient fine-tuning Proceedings of the AAAI Conference on Artificial Intelligence 2023 37 11 12799–12807

[30] Da-Wei Zhou and Qi-Wei Wang and Zhi-Hong Qi and Han-Jia Ye and De-Chuan Zhan and Ziwei Liu Deep class-incremental learning: A survey arXiv preprint arXiv:2302.03648 2023

[31] Da-Wei Zhou and Hai-Long Sun and Jingyi Ning and Han-Jia Ye and De-Chuan Zhan Continual Learning with Pre-Trained Models: A Survey Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24 2024 Kate Larson 8363–8371 International Joint Conferences on Artificial Intelligence Organization Survey Track 10.24963/ijcai.2024/924

[32] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521–3526, March 2017.

[33] Zhizhong Li and Derek Hoiem Learning without forgetting IEEE transactions on pattern analysis and machine intelligence 2017 40 12 2935–2947

[34] Arslan Chaudhry and Puneet K Dokania and Thalaiyasingam Ajanthan and Philip HS Torr Riemannian walk for incremental learning: Understanding forgetting and intransigence Proceedings of the European conference on computer vision (ECCV) 2018 532–547

[35] Xialei Liu and Marc Masana and Luis Herranz and Joost Van de Weijer and Antonio M Lopez and Andrew D Bagdanov Rotate your networks: Better weight consolidation and less catastrophic forgetting 2018 24th International Conference on Pattern Recognition (ICPR) 2018 2262–2268 IEEE

[36] Friedemann Zenke and Ben Poole and Surya Ganguli Continual learning through synaptic intelligence International conference on machine learning 2017 3987–3995 PMLR

[37] Rahaf Aljundi and Francesca Babiloni and Mohamed Elhoseiny and Marcus Rohrbach and Tinne Tuytelaars Memory aware synapses: Learning what (not) to forget Proceedings of the European conference on computer vision (ECCV) 2018 139–154

[38] Sang-Woo Lee and Jin-Hwa Kim and Jaehyun Jun and Jung-Woo Ha and Byoung-Tak Zhang Overcoming catastrophic forgetting by incremental moment matching Advances in neural information processing systems 2017 30

[39] Geoffrey Hinton and Oriol Vinyals and Jeff Dean Distilling the Knowledge in a Neural Network 2015

[40] Sylvestre-Alvise Rebuffi and Alexander Kolesnikov and Georg Sperl and Christoph H Lampert icarl: Incremental classifier and representation learning Proceedings of the IEEE conference on Computer Vision and Pattern Recognition 2017 2001–2010

[41] Prithviraj Dhar and Rajat Vikram Singh and Kuan-Chuan Peng and Ziyan Wu and Rama Chellappa Learning without Memorizing 2019

[42] Kibok Lee and Kimin Lee and Jinwoo Shin and Honglak Lee Overcoming catastrophic forgetting with unlabeled data in the wild Proceedings of the IEEE/CVF International Conference on Computer Vision 2019 312–321

[43] Saihui Hou and Xinyu Pan and Chen Change Loy and Zilei Wang and Dahua Lin Learning a unified classifier incrementally via rebalancing Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2019 831–839

[44] Prithviraj Dhar and Rajat Vikram Singh and Kuan-Chuan Peng and Ziyan Wu and Rama Chellappa Learning without memorizing Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2019 5138–5146

[45] Minsoo Kang, Jaeyoo Park, and Bohyung Han. Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation. In CVPR, 2022.

[46] Da-Wei Zhou and Han-Jia Ye and De-Chuan Zhan Co-transport for class-incremental learning Proceedings of the 29th ACM International Conference on Multimedia 2021 1645–1654

[47] Julio Hurtado and Alain Raymond-Sáez and Vladimir Araujo and Vincenzo Lomonaco and Alvaro Soto and Davide Bacciu Memory Population in Continual Learning via Outlier Elimination Proceedings of the IEEE/CVF International Conference on Computer Vision 2023 3481–3490

[48] Arthur Douillard and Matthieu Cord and Charles Ollion and Thomas Robert and Eduardo Valle PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning 2020

[49] Yaoyao Liu, Bernt Schiele, and Qianru Sun. Adaptive aggregation networks for class-incremental learning. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2021.

[50] Yaoyao Liu and Bernt Schiele and Qianru Sun RMM: Reinforced Memory Management for Class-Incremental Learning 2023

[51] Yue Wu and Yinpeng Chen and Lijuan Wang and Yuancheng Ye and Zicheng Liu and Yandong Guo and Yun Fu Large Scale Incremental Learning 2019

[52] Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.

[53] Arun Mallya and Dillon Davis and Svetlana Lazebnik Piggyback: Adapting a single network to multiple tasks by learning to mask weights Proceedings of the European conference on computer vision (ECCV) 2018 67–82

[54] Xiaotian Duan HAT-CL: A Hard-Attention-to-the-Task PyTorch Library for Continual Learning 2024

[55] Haeyong Kang and Rusty John Lloyd Mina and Sultan Rizky Hikmawan Madjid and Jaehong Yoon and Mark Hasegawa-Johnson and Sung Ju Hwang and Chang D. Yoo Forget-free Continual Learning with Winning Subnetworks Proceedings of the 39th International Conference on Machine Learning 2022 Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan 162 Proceedings of Machine Learning Research 10734–10750 PMLR

[56] Jin, Hyundong and Kim, Eunwoo Helpful or Harmful: Inter-task Association in Continual Learning Computer Vision – ECCV 2022 2022 Avidan, Shai and Brostow, Gabriel and Cissé, Moustapha and Farinella, Giovanni Maria and Hassner, Tal 519–535 Springer Nature Switzerland

[57] Arun Mallya and Svetlana Lazebnik PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning 2018

[58] Siavash Golkar and Michael Kagan and Kyunghyun Cho Continual Learning via Neural Pruning 2019

[59] Sangwon Jung and Hongjoon Ahn and Sungmin Cha and Taesup Moon Continual Learning with Node-Importance based Adaptive Group Sparse Regularization 2021

[60] Andrei A Rusu and Neil C Rabinowitz and Guillaume Desjardins and Hubert Soyer and James Kirkpatrick and Koray Kavukcuoglu and Razvan Pascanu and Raia Hadsell Progressive neural networks arXiv preprint arXiv:1606.04671 2016

[61] Chrisantha Fernando and Dylan Banarse and Charles Blundell and Yori Zwols and David Ha and Andrei A Rusu and Alexander Pritzel and Daan Wierstra Pathnet: Evolution channels gradient descent in super neural networks arXiv preprint arXiv:1701.08734 2017

[62] Tom Veniat and Ludovic Denoyer and Marc'Aurelio Ranzato Efficient Continual Learning with Modular Networks and Task-Driven Priors 2021

[63] David Lopez-Paz and Marc'Aurelio Ranzato Gradient Episodic Memory for Continual Learning 2022

[64] Arslan Chaudhry and Marc'Aurelio Ranzato and Marcus Rohrbach and Mohamed Elhoseiny Efficient Lifelong Learning with A-GEM 2019

[65] Shixiang Tang and Dapeng Chen and Jinguo Zhu and Shijie Yu and Wanli Ouyang Layerwise Optimization by Gradient Decomposition for Continual Learning 2021

[66] Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context-dependent processing in neural networks. Nature Machine Intelligence, 1(8):364–372, August 2019.

[67] Mehrdad Farajtabar and Navid Azizan and Alex Mott and Ang Li Orthogonal Gradient Descent for Continual Learning 2019

[68] Xiao Wang and Tianze Chen and Qiming Ge and Han Xia and Rong Bao and Rui Zheng and Qi Zhang and Tao Gui and Xuanjing Huang Orthogonal Subspace Learning for Language Model Continual Learning 2023

[69] Gobinda Saha and Isha Garg and Kaushik Roy Gradient Projection Memory for Continual Learning 2021

[70] Danruo Deng and Guangyong Chen and Jianye Hao and Qiong Wang and Pheng-Ann Heng Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning 2021

[71] Sen Lin and Li Yang and Deliang Fan and Junshan Zhang TRGP: Trust Region Gradient Projection for Continual Learning 2022

[72] Khurram Javed and Martha White Meta-Learning Representations for Continual Learning 2019

[73] Shawn Beaulieu and Lapo Frati and Thomas Miconi and Joel Lehman and Kenneth O. Stanley and Jeff Clune and Nick Cheney Learning to Continually Learn 2020

[74] Matthew Riemer and Ignacio Cases and Robert Ajemian and Miao Liu and Irina Rish and Yuhai Tu and Gerald Tesauro Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference 2019

[75] Jathushan Rajasegaran and Salman Khan and Munawar Hayat and Fahad Shahbaz Khan and Mubarak Shah iTAML: An Incremental Task-Agnostic Meta-learning Approach 2020

[76] Gunshi Gupta and Karmesh Yadav and Liam Paull La-MAML: Look-ahead Meta Learning for Continual Learning 2020

[77] Massimo Caccia and Pau Rodriguez and Oleksiy Ostapenko and Fabrice Normandin and Min Lin and Lucas Caccia and Issam Laradji and Irina Rish and Alexandre Lacoste and David Vazquez and Laurent Charlin Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning 2021

[78] K J Joseph and Vineeth N Balasubramanian Meta-Consolidation for Continual Learning 2020

[79] Julio Hurtado and Alain Raymond-Saez and Alvaro Soto Optimizing Reusable Knowledge for Continual Learning via Metalearning 2021

[80] Jeremy Howard and Sebastian Ruder Universal Language Model Fine-tuning for Text Classification 2018

[81] Emma Strubell and Ananya Ganesh and Andrew McCallum Energy and Policy Considerations for Deep Learning in NLP 2019

[82] James Kirkpatrick and Razvan Pascanu and Neil Rabinowitz and Joel Veness and Guillaume Desjardins and Andrei A Rusu and Kieran Milan and John Quan and Tiago Ramalho and Agnieszka Grabska-Barwinska and others Overcoming catastrophic forgetting in neural networks Proceedings of the national academy of sciences 2017 114 13 3521–3526

[83] Junxian He and Chunting Zhou and Xuezhe Ma and Taylor Berg-Kirkpatrick and Graham Neubig Towards a unified view of parameter-efficient transfer learning arXiv preprint arXiv:2110.04366 2021

[84] Mengjie Zhao and Tao Lin and Fei Mi and Martin Jaggi and Hinrich Schütze Masking as an efficient alternative to finetuning for pretrained language models arXiv preprint arXiv:2004.12406 2020

[85] Neil Houlsby and Andrei Giurgiu and Stanislaw Jastrzebski and Bruna Morrone and Quentin De Laroussilhe and Andrea Gesmundo and Mona Attariyan and Sylvain Gelly Parameter-Efficient Transfer Learning for NLP Proceedings of the 36th International Conference on Machine Learning 2019 Chaudhuri, Kamalika and Salakhutdinov, Ruslan 97 Proceedings of Machine Learning Research 2790–2799 PMLR

[86] Yi Xin and Siqi Luo and Haodi Zhou and Junlong Du and Xiaohong Liu and Yue Fan and Qing Li and Yuntao Du Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey 2024

[87] Shoufa Chen and Chongjian Ge and Zhan Tong and Jiangliu Wang and Yibing Song and Jue Wang and Ping Luo AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition 2022

[88] Jonas Pfeiffer and Aishwarya Kamath and Andreas Rücklé and Kyunghyun Cho and Iryna Gurevych AdapterFusion: Non-Destructive Task Composition for Transfer Learning 2021

[89] Shwai He and Run-Ze Fan and Liang Ding and Li Shen and Tianyi Zhou and Dacheng Tao MerA: Merging Pretrained Adapters For Few-Shot Learning 2023

[90] Yaqing Wang and Sahaj Agarwal and Subhabrata Mukherjee and Xiaodong Liu and Jing Gao and Ahmed Hassan Awadallah and Jianfeng Gao AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning 2022

[91] Edward J. Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen LoRA: Low-Rank Adaptation of Large Language Models 2021

[92] Brian Lester and Rami Al-Rfou and Noah Constant The Power of Scale for Parameter-Efficient Prompt Tuning 2021

[93] Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. Gpt understands, too. AI Open, 5:208–215, 2024.

[94] Xiang Lisa Li and Percy Liang Prefix-Tuning: Optimizing Continuous Prompts for Generation 2021

[95] Zhen Wang and Rameswar Panda and Leonid Karlinsky and Rogerio Feris and Huan Sun and Yoon Kim Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning 2023

[96] Tim Dettmers and Artidoro Pagnoni and Ari Holtzman and Luke Zettlemoyer QLoRA: Efficient Finetuning of Quantized LLMs 2023

[97] Shih-Yang Liu and Chien-Yi Wang and Hongxu Yin and Pavlo Molchanov and Yu-Chiang Frank Wang and Kwang-Ting Cheng and Min-Hung Chen DoRA: Weight-Decomposed Low-Rank Adaptation 2024

[98] Mahdi Nikdan and Soroush Tabesh and Elvir Crnčević and Dan Alistarh RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation 2024

[99] Dawid J. Kopiczko and Tijmen Blankevoort and Yuki M. Asano VeRA: Vector-based Random Matrix Adaptation 2024

[100] Jonathan Frankle and Michael Carbin The lottery ticket hypothesis: Finding sparse, trainable neural networks arXiv preprint arXiv:1803.03635 2018

[101] Yinhan Liu and Myle Ott and Naman Goyal and Jingfei Du and Mandar Joshi and Danqi Chen and Omer Levy and Mike Lewis and Luke Zettlemoyer and Veselin Stoyanov Roberta: A robustly optimized bert pretraining approach arXiv preprint arXiv:1907.11692 2019

[102] Victor Sanh and Lysandre Debut and Julien Chaumond and Thomas Wolf DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter arXiv preprint arXiv:1910.01108 2019

[103] Elad Ben Zaken and Shauli Ravfogel and Yoav Goldberg Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models arXiv preprint arXiv:2106.10199 2021

[104] Alex Wang and Amanpreet Singh and Julian Michael and Felix Hill and Omer Levy and Samuel R Bowman GLUE: A multi-task benchmark and analysis platform for natural language understanding arXiv preprint arXiv:1804.07461 2018

[105] Haokun Liu and Derek Tam and Mohammed Muqeeth and Jay Mohta and Tenghao Huang and Mohit Bansal and Colin A Raffel Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning Advances in Neural Information Processing Systems 2022 35 1950–1965

[106] Joan Serra and Didac Suris and Marius Miron and Alexandros Karatzoglou Overcoming catastrophic forgetting with hard attention to the task International conference on machine learning 2018 4548–4557 PMLR

[107] Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, and Lei Shu. Achieving forgetting prevention and knowledge transfer in continual learning. ArXiv, abs/2112.02706, 2021.

[108] James Seale Smith and Leonid Karlinsky and Vyshnavi Gutta and Paola Cascante-Bonilla and Donghyun Kim and Assaf Arbelle and Rameswar Panda and Rogerio Feris and Zsolt Kira Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 11909–11919

[109] Anastasia Razdaibiedina and Yuning Mao and Rui Hou and Madian Khabsa and Mike Lewis and Amjad Almahairi Progressive Prompts: Continual Learning for Language Models International Conference on Learning Representations 2023

[110] Xinyuan Gao and Songlin Dong and Yuhang He and Qiang Wang and Yihong Gong Beyond Prompt Learning: Continual Adapter for Efficient Rehearsal-Free Continual Learning 2024

[111] Wentao Zhang and Yujun Huang and Tong Zhang and Qingsong Zou and Wei-Shi Zheng and Ruixuan Wang Adapter Learning in Pretrained Feature Extractor for Continual Learning of Diseases 2023

[112] Jiazuo Yu and Yunzhi Zhuge and Lu Zhang and Ping Hu and Dong Wang and Huchuan Lu and You He Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters 2024

[113] Huiyi Wang and Haodong Lu and Lina Yao and Dong Gong Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning 2024

[114] Qiangpu Chen, Jiahua Huang, Wushao Wen, Qingling Li, Rumin Zhang, and Jinghui Qin. Cat: Continual adapter tuning for aspect sentiment classification. Neurocomputing, 580:127423, 2024.

[115] Zhi-Hong Qi and Da-Wei Zhou and Yiran Yao and Han-Jia Ye and De-Chuan Zhan Adaptive Adapter Routing for Long-Tailed Class-Incremental Learning 2024

[116] Hong Li and Zhiquan Tan and Xingyu Li and Weiran Huang ATLAS: Adapter-Based Multi-Modal Continual Learning with a Two-Stage Learning Strategy 2024

[117] Yujun Huang and Wentao Zhang and Ruixuan Wang Expand and Merge: Continual Learning with the Guidance of Fixed Text Embedding Space 2024 International Joint Conference on Neural Networks (IJCNN) 2024 1-8 10.1109/IJCNN60899.2024.10650910

[118] Yan-Shuo Liang and Wu-Jun Li InfLoRA: Interference-Free Low-Rank Adaptation for Continual Learning 2024

[119] Haiyang Guo and Fei Zhu and Wenzhuo Liu and Xu-Yao Zhang and Cheng-Lin Liu Federated Class-Incremental Learning with Prototype Guided Transformer arXiv preprint arXiv:2401.02094 2024

[120] Huancheng Chen and Jingtao Li and Nidham Gazagnadou and Weiming Zhuang and Chen Chen and Lingjuan Lyu Dual Low-Rank Adaptation for Continual Learning with Pre-Trained Models 2024

[121] Zeinab Borhanifard, Heshaam Faili, and Yadollah Yaghoobzadeh. Combining replay and lora for continual learning in natural language understanding. Computer Speech & Language, 90:101737, 2025.

[122] Rajas Chitale, Ankit Vaidya, Aditya Kane, and Archana Santosh Ghotkar. Task arithmetic with loRA for continual learning. In Workshop on Advancing Neural Network Training: Computational Efficiency, Scalability, and Resource Optimization (WANT@NeurIPS 2023), 2023.

[123] James Seale Smith, Yen-Chang Hsu, Zsolt Kira, Yilin Shen, and Hongxia Jin. Continual diffusion with stamina: Stack-and-mask incremental adapters. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1744–1754, 2023.

[124] Weijieying Ren and Xinlong Li and Lei Wang and Tianxiang Zhao and Wei Qin Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning 2024

[125] Muhammad Gul Zain Ali Khan and Muhammad Ferjad Naeem and Luc Van Gool and Didier Stricker and Federico Tombari and Muhammad Zeshan Afzal Introducing language guidance in prompt-based continual learning Proceedings of the IEEE/CVF International Conference on Computer Vision 2023 11463–11473

[126] Alex Krizhevsky and Geoffrey Hinton and others Learning multiple layers of features from tiny images 2009

[127] Jingyang Qiao and Xin Tan and Chengwei Chen and Yanyun Qu and Yong Peng and Yuan Xie and others Prompt Gradient Projection for Continual Learning The Twelfth International Conference on Learning Representations

[128] Gobinda Saha and Isha Garg and Kaushik Roy Gradient Projection Memory for Continual Learning International Conference on Learning Representations

[129] Yabin Wang and Zhiwu Huang and Xiaopeng Hong S-prompts learning with pre-trained transformers: An occam’s razor for domain incremental learning Advances in Neural Information Processing Systems 2022 35 5682–5695

[130] Zhiyuan Hu and Jiancheng Lyu and Dashan Gao and Nuno Vasconcelos POP: Prompt Of Prompts for Continual Learning arXiv preprint arXiv:2306.08200 2023

[131] David Lopez-Paz and Marc'Aurelio Ranzato Gradient episodic memory for continual learning Advances in neural information processing systems 2017 30

[132] Natalia Díaz-Rodríguez and Vincenzo Lomonaco and David Filliat and Davide Maltoni Don't forget, there is more than forgetting: new metrics for Continual Learning arXiv preprint arXiv:1810.13166 2018

[133] Kuan-Ying Lee and Yuanyi Zhong and Yu-Xiong Wang Do pre-trained models benefit equally in continual learning? Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision 2023 6485–6493

[134] Daisuke Niizumi and Daiki Takeuchi and Yasunori Ohishi and Noboru Harada and Kunio Kashino BYOL for audio: Exploring pre-trained general-purpose audio representations IEEE/ACM Transactions on Audio, Speech, and Language Processing 2022 31 137–151

[135] Huaishao Luo and Lei Ji and Botian Shi and Haoyang Huang and Nan Duan and Tianrui Li and Jason Li and Taroon Bharti and Ming Zhou Univl: A unified video and language pre-training model for multimodal understanding and generation arXiv preprint arXiv:2002.06353 2020

[136] Wenxuan Ma and Shuang Li and Jingxuan Kang and others Language Semantic Graph Guided Data-Efficient Learning Advances in Neural Information Processing Systems 2024 36

[137] Richard Socher and Milind Ganjoo and Christopher D Manning and Andrew Ng Zero-shot learning through cross-modal transfer Advances in neural information processing systems 2013 26

[138] Gabriel Ilharco and Marco Tulio Ribeiro and Mitchell Wortsman and Suchin Gururangan and Ludwig Schmidt and Hannaneh Hajishirzi and Ali Farhadi Editing Models with Task Arithmetic 2023

[139] Prateek Yadav and Derek Tam and Leshem Choshen and Colin Raffel and Mohit Bansal TIES-Merging: Resolving Interference When Merging Models 2023

[140] Le Yu and Bowen Yu and Haiyang Yu and Fei Huang and Yongbin Li Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch 2024

[141] Daniel Marczak and Bartłomiej Twardowski and Tomasz Trzciński and Sebastian Cygert MagMax: Leveraging Model Merging for Seamless Continual Learning 2024

[142] Enneng Yang and Li Shen and Zhenyi Wang and Guibing Guo and Xiaojun Chen and Xingwei Wang and Dacheng Tao Representation Surgery for Multi-Task Model Merging 2024

[143] Eric Nuertey Coleman, Luigi Quarantiello, Julio Hurtado, and Vincenzo Lomonaco. Adaptive LoRA merging for efficient domain incremental learning. In Adaptive Foundation Models: Evolving AI for Personalized and Efficient Learning, 2024.

[144] Anke Tang and Enneng Yang and Li Shen and Yong Luo and Han Hu and Bo Du and Dacheng Tao Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging 2025

[145] Neil Houlsby and Andrei Giurgiu and Stanislaw Jastrzebski and Bruna Morrone and Quentin de Laroussilhe and Andrea Gesmundo and Mona Attariyan and Sylvain Gelly Parameter-Efficient Transfer Learning for NLP 2019

[146] Rylan Schaeffer and Brando Miranda and Sanmi Koyejo Are emergent abilities of large language models a mirage? Advances in Neural Information Processing Systems 2024 36

[147] Zhiqiu Lin and Jia Shi and Deepak Pathak and Deva Ramanan The clear benchmark: Continual learning on real-world imagery Thirty-fifth conference on neural information processing systems datasets and benchmarks track (round 2) 2021

[148] Eli Verwimp and Kuo Yang and Sarah Parisot and Lanqing Hong and Steven McDonagh and Eduardo Pérez-Pellitero and Matthias De Lange and Tinne Tuytelaars Clad: A realistic continual learning benchmark for autonomous driving Neural Networks 2023 161 659–669

[149] Kenneth Marino and Xinlei Chen and Devi Parikh and Abhinav Gupta and Marcus Rohrbach Krisp: Integrating implicit and symbolic knowledge for open-domain knowledge-based vqa Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2021 14111–14121

[150] Kexin Yi and Jiajun Wu and Chuang Gan and Antonio Torralba and Pushmeet Kohli and Josh Tenenbaum Neural-symbolic vqa: Disentangling reasoning from vision and language understanding Advances in neural information processing systems 2018 31

[151] Stan Weixian Lei and Difei Gao and Jay Zhangjie Wu and Yuxuan Wang and Wei Liu and Mengmi Zhang and Mike Zheng Shou Symbolic replay: Scene graph as prompt for continual learning on vqa task Proceedings of the AAAI Conference on Artificial Intelligence 2023 37 1 1250–1259

[152] Honghua Dong and Jiayuan Mao and Tian Lin and Chong Wang and Lihong Li and Denny Zhou Neural logic machines arXiv preprint arXiv:1904.11694 2019

[153] Sumith Kulal and Jiayuan Mao and Alex Aiken and Jiajun Wu Hierarchical motion understanding via motion programs Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2021 6568–6576

[154] Yunchao Liu and Zheng Wu Learning to describe scenes with programs International conference on learning representations 2019

[155] Xi Zhang and Feifei Zhang and Changsheng Xu Vqacl: A novel visual question answering continual learning setting Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 19102–19112

[156] Emanuele Marconato and Gianpaolo Bontempo and Elisa Ficarra and Simone Calderara and Andrea Passerini and Stefano Teso Neuro-symbolic continual learning: Knowledge, reasoning shortcuts and concept rehearsal arXiv preprint arXiv:2302.01242 2023

[157] Robin Manhaeve and Sebastijan Dumancic and Angelika Kimmig and Thomas Demeester and Luc De Raedt Deepproblog: Neural probabilistic logic programming Advances in neural information processing systems 2018 31

[158] Eli Verwimp and Shai Ben-David and Matthias Bethge and Andrea Cossu and Alexander Gepperth and Tyler L Hayes and Eyke Hüllermeier and Christopher Kanan and Dhireesha Kudithipudi and Christoph H Lampert and others Continual learning: Applications and the road forward arXiv preprint arXiv:2311.11908 2023

[159] Ameya Prabhu and Zhipeng Cai and Puneet Dokania and Philip Torr and Vladlen Koltun and Ozan Sener Online continual learning without the storage constraint arXiv preprint arXiv:2305.09253 2023

[160] Zheda Mai and Ruiwen Li and Jihwan Jeong and David Quispe and Hyunwoo Kim and Scott Sanner Online continual learning in image classification: An empirical survey Neurocomputing 2022 469 28–51

[161] Rahaf Aljundi and Klaas Kelchtermans and Tinne Tuytelaars Task-free continual learning Proceedings of the IEEE/CVF conference on computer vision and pattern recognition 2019 11254–11263

[162] Benedikt Bagus and Alexander Gepperth and Timothée Lesort Beyond supervised continual learning: a review arXiv preprint arXiv:2208.14307 2022

[163] Yanbei Chen and Massimiliano Mancini and Xiatian Zhu and Zeynep Akata Semi-supervised and unsupervised deep visual learning: A survey IEEE transactions on pattern analysis and machine intelligence 2022