E-CGL: an efficient continual graph learner

Abstract

Continual learning (CL) has emerged as a crucial paradigm for learning from sequential data while retaining previous knowledge. Continual graph learning (CGL), characterized by dynamically evolving graphs from streaming data, presents distinct challenges that demand efficient algorithms to prevent catastrophic forgetting. The first challenge stems from the interdependencies between different graph data, in which previous graphs influence new data distributions. The second challenge is handling large graphs in an efficient manner. To address these challenges, we propose an efficient continual graph learner (E-CGL) in this paper. We address the interdependence issue by demonstrating the effectiveness of replay strategies and introducing a combined sampling approach that considers both node importance and diversity. To improve efficiency, E-CGL leverages a simple yet effective multi-layer perceptron (MLP) model that shares weights with a graph neural network (GNN) during training, thereby accelerating computation by circumventing the expensive message-passing process. Our method achieves state-of-the-art results on four CGL datasets under two settings, while significantly lowering the catastrophic forgetting value to an average of −1.1%. Additionally, E-CGL achieves the training and inference speedup by an average of 15.83× and 4.89×, respectively, across four datasets. These results indicate that E-CGL not only effectively manages correlations between different graph data during continual training but also enhances efficiency in large-scale CGL.

1. Introduction

Graphs have garnered significant research attention due to their ubiquity in modeling relational data (Kipf and Welling, 2017; Veličković et al., 2018; Xu et al., 2019). In real-world applications, graph data continuously expand, and new patterns emerge over time. For instance, recommendation networks may introduce new product categories, citation networks may see the rise of novel research topics, and chemical design may discover new molecules and drugs. To adapt to these evolving scenarios and provide up-to-date predictions, graph models must constantly learn and update. However, traditional training strategies (Niepert et al., 2016; Kipf and Welling, 2017) experience catastrophic forgetting (Yuan et al., 2023) when adapting to new data, resulting in poor performance on previous tasks.

As a result, methods that can rapidly adapt to new classes while maintaining performance on previous tasks are required. Continual learning (CL, also known as incremental learning or lifelong learning) aims to do this by learning from new data while retaining prior knowledge (Thrun, 1994). While CL has been extensively studied in areas such as computer vision (Rebuffi et al., 2017; Buzzega et al., 2020; Ni et al., 2023) and natural language processing (Srinivasan et al., 2022; Fan et al., 2022), its application to graph-structured data presents unique challenges for a diverse set of downstream tasks, including supervised node classification and unsupervised dynamic network embedding (Choi et al., 2024; Gao et al., 2024; Wang ZZ et al., 2025).

The first challenge arises from the topological interdependence of graph snapshots, in which new tasks inherit structural and attributive dependencies from previous tasks. Topological interdependence occurs when sequential graph snapshots {G1, G2, ..., GT} share nodes/edges or have attribute continuity, resulting in conditional dependencies between their distributions. Specifically, in continual graph learning (CGL), the conditional likelihood of observing Gt depends on previous graphs: p(θ|G1:t) ∝ p(Gt|G1:t−1, θ)p(θ|G1:t−1)/p(Gt), where p(θ|G1:t) refers to the posterior distribution of the model parameter θ after observing all snapshots from round 1 to t, and p(θ|G1:t−1) is an estimate of θ when only the first t−1 snapshots are observed.