MaskDGNN: Self-Supervised Dynamic Graph Neural Networks with Activeness-aware Temporal Masking

1 Introduction

Dynamic graphs are prevalent in real-world systems, capturing the evolving relationships and interactions within complex networks [Kazemi et al., 2020]. For instance, in social networks [Song et al., 2019; Ma et al., 2020], they model dynamic user interactions such as friendship formation and information diffusion. In transportation networks [Kumar et al., 2019; Wu et al., 2019b], they reflect real-time traffic patterns, with nodes and edges representing locations and their changing connectivity. Similarly, in e-commerce networks [Sharma et al., 2023; Zhang et al., 2022], they track temporal shifts in user behaviors, such as browsing, purchasing, and reviewing products. These networks are characterized by temporally evolving nodes and edges, exhibiting rich temporal features that are essential for understanding network dynamics. Analyzing these properties provides valuable insights into the behavior and evolution of complex systems, enabling more accurate predictions and informed decision-making.

∗Yanwei Yu is the corresponding author.

Compared to static graph methods [You et al., 2021; Hassani and Khasahmadi, 2020; Yu et al., 2022; Fu et al., 2023], dynamic graph approaches are more adept at capturing the temporal evolution of nodes and edges, offering enhanced expressive power for modeling complex realworld interactions. Existing dynamic graph neural networks (DGNNs) [Xue et al., 2022] can be broadly categorized into continuous and discrete models. On the one hand, continuous DGNNs [Tian et al., 2023; Rossi et al., 2020; Cong et al., 2023; Poursafaei et al., 2022] capture finegrained temporal information by transforming dynamic graph data into time series and employing temporal encoders, such as Transformers, to model dynamic patterns [Yu et al., 2023; Xu et al., 2020]. These approaches excel in capturing detailed temporal dynamics by leveraging powerful temporal encoders. Discrete DGNNs [Liu et al., 2021], on the other hand, employ more lightweight designs, focusing on capturing dynamic dependencies without relying on continuous time series. Early models such as EvolveGCN [Pareja et al., 2020], DGNN [Manessi et al., 2020], and DySAT [Sankar et al., 2020] utilize recurrent networks or temporal encoders to capture the temporal evolution of graphs. More recent models, including Roland [You et al., 2022], WinGNN [Zhu et al., 2023] and DyTed [Zhang et al., 2023], replace these traditional temporal encoders with techniques such as metalearning [Finn et al., 2017], sliding window strategies, and disentangling methods. These approaches offer significant improvements in performance, enabling real-time processing while maintaining high predictive accuracy.

Despite significant advancements in dynamic graph learning, two key challenges persist: (1) Existing methods tend to overlook the negative impact of redundant information on dynamic graphs. Many methods rely on extensive historical information, overemphasizing low-activeness nodes and infrequently changing edges which may lead to redundant information. This reduces the model's ability to focus on key interactions and harms its predictive performance. Emphasizing the dynamic changes of highly active and critical nodes is crucial for improving learning effectiveness. (2) Most existing methods struggle in addressing the distribution shifting issue on dynamic graphs. The temporal structure distribution in dynamic graphs evolves over time, caused by changes in node relationships. For instance, in a Bitcoin trading network, traders typically follow certain patterns based on market trends, but a sudden shift in focus, such as a major news event or a market crash, can dramatically alter trading behaviors. Existing models struggle to adapt to these shifts, hindering their ability to capture evolving interactions and compromising predictive accuracy.

To address these challenges, we propose MaskDGNN, a novel dynamic graph neural network that reduces the negative impact of redundant information and captures intrinsic features under distribution shifting. MaskDGNN consists of two main components: (1) Self-supervised activeness-aware temporal masking module: It identifies and retains edges between highly active nodes while masking edges between nodes with low activeness. By comparing successive graph snapshots, this strategy filters out redundant information, ensuring the model focuses on the most relevant interactions, thereby improving predictive accuracy. (2) Adaptive frequency enhancing graph representation learner: This module addresses the feature capture issue under the temporal structure distribution shifting challenge in dynamic graphs by leveraging Fourier transforms to enhance the graph's temporal features. It identifies and emphasizes the key frequency components in active node representations, allowing the model to better adapt to distribution shifts, such as sudden changes in trading patterns or behaviors. Additionally, a sliding window strategy aggregates long-term dependencies, improving robustness and capturing macro-level structural changes across graph snapshots. Our contributions are summarized as follows:

• We propose MaskDGNN, a novel dynamic graph neural network architecture that integrates a self-supervised activeness-aware temporal masking module with an adaptive frequency enhancing graph representation learner. This design reduces redundant information while capturing intrinsic features under distribution shifting, significantly improving link prediction accuracy.
• We introduce an activeness-aware masking mechanism that compares each node's activeness between consecutive time steps, retaining edges of highly active nodes and masking low-activity ones, effectively filtering out redundancy and focusing on key temporal interactions.
• Extensive experiments on five real-world dynamic graph datasets show that MaskDGNN outperforms state-of-theart methods, with average improvements of 7.07%, 2.62%, 13.87%, and 7.31% in Accuracy, AUC, MRR, and Recall@10, respectively.

2 RelatedWork

2.1 Dynamic Graph Neural Networks

Research on DGNNs [Yang et al., 2024; Kazemi et al., 2020] mainly focuses on capturing the dynamic changes in graph structures and their temporal dependencies to enhance the ability of dynamic graph representation learning. Existing DGNNs can be classified into two types based on how time is recorded: discrete DGNNs and continuous DGNNs

[Wang et al., 2021; Jin et al., 2022; Kachole et al., 2023; Trivedi et al., 2019]. This paper primarily discusses the related work in discrete DGNNs. Early works such as EvolveGCN [Pareja et al., 2020] and DGNN [Manessi et al., 2020] leverage the combination of recurrent networks (GRU [Cho et al., 2014] or LSTM [Hochreiter and Schmidhuber, 1997]) with GCN parameters and the integration of long short-term memory networks with graph convolutional networks. DGT [Cong et al., 2021] and DySAT [Sankar et al., 2020] are based on Transformers [Vaswani, 2017] and are designed to capture dynamic graph information. MTSN [Liu et al., 2021] models the local higher-order structures and temporal evolution of dynamic networks by learning the motif structure of the graph. Dyngraph2vec [Goyal et al., 2020] adopts a deep architecture combining dense and recurrent layers to learn temporal dynamics in dynamic graphs. Recent studies further optimized dynamic graph modeling frameworks. For instance, DyTed [Zhang et al., 2023] proposes a disentangled representation learning approach that leverages contrastive learning to distinguish between temporal and invariant features. DRLAN [Liu et al., 2020] introduces a hybrid offlineonline architecture for updating node embeddings in largescale attributed dynamic networks. ROLAND [You et al., 2022] is a general framework from the perspective of metalearning, enabling the transformation of static GNNs into dynamic GNNs. WinGNN [Zhu et al., 2023] utilizes a sliding window mechanism to reduce parameter size, alleviating the computational and storage burdens of long sequences. SILD [Zhang et al., 2024] employs Fourier transforms to achieve different frequency separation. STDGL [Liu et al., 2024b] designs time-aware auxiliary tasks that incorporate local and global connectivity information.

2.2 Self-SupervisedMasked GraphModels

With the growing use of graph neural networks (GNNs) in areas like social networks, recommendation systems, and bioinformatics, graph masking strategies become essential for highlighting key structures during training. These strategies mask certain nodes or edges to guide the model in focusing on critical graph information and reconstructing the masked parts, improving its understanding of graph structures. Initially introduced in computer vision by MAE [He et al., 2022], the concept is extended in GraphMAE [Hou et al., 2022], which masks node features to generate compact graph representations. S2GAE [Tan et al., 2023] integrates masking with graph autoencoders, setting the stage for future developments. MaskGAE [Li et al., 2023] enhances graph structure capture by using random walks to determine masking positions, while Bandana [Zhao et al., 2024] introduces edge masking with a continuous range [0, 1], balancing structural preservation and flexibility. Finally, StructMAE [Liu et al., 2024a] employs progressive masking based on structural scoring to improve the model's ability to reconstruct key graph structures.

3 Preliminaries

Definition 1 (Discrete-Time Dynamic Graphs). Given a dynamic graph dataset, we define a discrete-time dynamic graph as a sequence of snapshots G = {G1 ,G2 , . . . ,GT }, where T denotes the total number of snapshots. Each snapshot at time t, i.e., Gt = (V t , E t ), consists of a node set V t and an edge set E t ⊆ V t × V t . The adjacency matrix At ∈ R|V t |×|V t | represents the edges in E t , and Xt ∈ R|V t |×f denotes the node feature matrix at time t, where f is the dimensionality of the node features.

Problem. The task of link prediction involves learning a function f that predicts the presence of an unobserved edge at time t + 1, based on the node features and graph structure from the previous snapshots:

4 Methodology

In this section, we provide a detailed description of our proposed MaskDGNN. The overall architecture of MaskDGNN is shown in Fig. 1, which consists of two key components: (1) Self-Supervised Activeness-aware Temporal Masking Module, and (2) Adaptive Frequency Enhancing Graph Representation Learner.

4.1 Self-Supervised Activeness-aware Temporal Masking Module

We propose a temporal masking mechanism that dynamically evaluates node activeness at each time step and selectively masks less active edges. This approach enhances dynamic graph representation learning by preserving critical structural updates while filtering out redundant information.

Node Activeness Score Calculation

To better focus on key and active nodes while minimizing interference from redundant nodes, we introduce a node activeness score Atvi to evaluate each node. This score determines edge retention or masking: edges between high-activeness nodes are retained, while those between low-activeness nodes are masked with a certain probability. Specifically, Atvi is defined as the sum of two components: the node dynamics score Dtvi and the node significance score S tvi .

The node dynamics score Dtvi quantifies the extent of changes in a node's edges over time. It is computed by comparing the node's edge changes between the current graph Gt and the previous time step graph Gt-1 , along with the ratio of the node degree at the previous time step. The specific definition is as follows:

where vi ∈ Gt , and the calculation of the node dynamics score Dtvi depends on whether vi exists in Gt or not. deg(vi)t-1 denotes the degree of node vi in Gt-1 , η and γ are introduced to control the weight of the dynamics score for existing and newly added nodes, λ is a parameter that adjusts the influence of ∆|N t vi | on the dynamics score, and ∆|N t vi | is calculated as:

where ⊕ denotes the symmetric difference operator, N t-1 vi and N t vi are the edge sets of node vi at time step t - 1 and t, respectively.

The node significance score S tvi measures the influence of node vi in the current graph Gt , which is computed using the PageRank algorithm Rank(·) to evaluate the global importance of each node within the graph:

where d is the damping factor (typically set to 0.85), and N is the total number of nodes. The first term models random jumps to any node in the graph, while the second term aggregates contributions from the node's neighbors.

Finally, the node dynamics score Dtvi and the node significance score S tvi are weighted and fused by control parameter α, resulting in a comprehensive node activeness score Atvi . This score integrates both node dynamics and global importance, providing a more holistic representation of the node's significance in the dynamic graph as follows:

Self-Supervised Temporal Masking

Self-supervised graph masking has been widely explored for static graphs, where masked structures help capture essential features and patterns. Building on this, we propose a selfsupervised temporal masking module for dynamic graphs, using node activeness scores to guide the masking of redundant edges. Specifically, the mask probability pteij for the edge between nodes vi and vj at time step t is defined as follows:

where Atvi and Atvj represent the activeness scores of nodes vi and vj, and Mean(·) represents the mean pooling operation. Edges with low pteij can be considered redundant and are more likely to be masked.

To ensure that the total number of masked edges aligns with the desired masking ratio, the adjusted mask probability ˆ pteij for each edge is derived by scaling the original mask probability:

where Nedges is the total number of edges, and p is the overall proportion of masked edges in the graph Gt . The final decision to mask each edge at time t is determined by the adjusted mask probability ˆ pteij :

where Uij is a random variable following a Bernoulli distribution B(ˆ pteij ). If Uij = 1, the edge etij is not masked (Mask(etij ) = 0); if Uij = 0, the edge etij is masked (Mask(etij ) = 1).

After masking operation on each snapshot, we obtain the masked graph set Gm = {G1m,G2m, . . . ,GTm} and the retained graphs set Gr = {G1r ,G2r . . . ,GTr } , noted that Gtm ∪ Gtr = Gt .

Unlike previous models directly discard masked edges, The self-supervised loss for the masked edges treats them as positive samples, preserving the link information in the original graph and ensuring the model comprehends the overall graph structure.Ltssl = - ∑

where E tm represents the set of masked edges at time t, and treated as positive samples in the training process. Here, ytu,v is the ground-truth label for the edge eij at time t, and ˆ ytu,v is the predicted result.

4.2 Adaptive Frequency Enhancing Graph Representation Learner

We propose this module to enhance dynamic graph representations through spectral-temporal processing. It transforms node embeddings via FFT, applies learnable frequency filters to capture intrinsic features under distribution shifting, and employs a sliding window to capture long-term dependencies through gradient aggregation.

Graph Convolutional Encoder

We feed the retained graph Gr into a l-layer decoupled GCN [Wu et al., 2019a] to ensure the model focuses on the more critical information:

where H(l) t ∈ Rn×d is the representation of the nodes derived at l-th layer at time t, ˆA = D-1/2AD-1/2 denotes the normalized adjacent matrix given that A is the adjacent matrix with self-loops and D is the corresponding diagonal degree matrix, andW(l) is a learnable parameter matrix at l-th layer.

Adaptive Frequency Enhancing Layer

Inspired by FreeDyG [Tian et al., 2023], we enhance key frequency components in the graph representations using Fourier-based modulation. Given the l-th layer GNN output H(l) t ∈ RL×d at time t, we first apply the 1D Fast Fourier Transform (FFT) along the temporal dimension:

where F denotes the 1D FFT, and H(l) t ∈ C⌈ L2 +1⌉×d represents the frequency components of H(l) t , with C denoting the complex number domain.

Next, we adaptively enhance the frequency components by multiplying them element-wise with a learnable complex number tensor W ∈ C⌈ L2 +1⌉×d :

where (·) represents element-wise multiplication, and ̂H(l) t ∈ C⌈ L2 +1⌉×d denotes the enhanced frequency components.

Finally, we apply the inverse FFT to transform the enhanced frequency components ̂H(l) t back into the time domain:

where F -1 denotes the inverse 1D FFT, converting the complex number tensor back into a real number tensor. Then, a residual connection and dropout are applied to the resulting tensor:

Then, we predict the probability of edge from node u to node v through a multi-layer perceptron (MLP):

where ˆ ytu,v is the predicted label at time t, h(l) u,t and h(l) v,t are the feature embeddings of nodes u and v, obtained from Hlt .

Sliding Window Gradient Aggregation

Inspired by WinGNN [Zhu et al., 2023], we also adopt a sliding window gradient aggregation strategy to model long-term dependencies in dynamic graphs. This strategy utilizes two parameter layers: inter-window parameters Θwi and intrawindow parameters θwi t . At each time step t, the inner parameters θwi t are updated using snapshot-specific gradients, while the outer parameters Θwi are updated by aggregating the inner gradients at the end of each window wi. Based on the label data ytu,v on Gt within window wi, we can obtain the train loss via cross-entropy loss:

where Ltssl is the self-supervised loss term, which includes contributions from the masked edges E tm, treated as positive samples. The second term computes the loss for the retained edges E t r , using true labels ytu,v and predicted probabilities ˆ ytu,v . The third term calculates the loss for negative samples E t neg . The gradient aggregation process is defined as:

where θwi t represents the model parameters at time step t within sliding window wi, Lwi Gt is the loss function for the snapshot at time t within wi, and τ is the learning rate for θ.

The gradient aggregation across multiple snapshots in the sliding window wi is computed as follows:

where ∇Lwi denotes the aggregated gradient over the window, and ζt represents the weight assigned to each snapshot.

For the next window, the inter-window parameters Θwi+1 are updated as follows:

where β is the learning rate for Θ, and Θwi represents the outer parameters in window i.

5 Experiments

In this section, we perform model evaluation to investigate the effectiveness of our MaskDGNN 1 and baseline methods on five real-world datasets. Our experiments aim to answer the research questions as follows:

• RQ1: What is the performance of our MaskDGNN as compared to various state-of-the-art social relationship inference methods?
• RQ2: How do the key components contribute to the performance?
• RQ3: How do the key hyperparameters influence the performance of the proposed MaskDGNN?

5.1 Datasets

In our experiments, we evaluate the performance of all methods using five publicly available real-world datasets: BitcoinAlpha, Bitcoin-OTC [Kumar et al., 2016], UCI-Message [Panzarasa et al., 2009], MOOC [Kumar et al., 2019], and Wiki-Talk [De Rijke, 2017]. Notably, Wiki-Talk is a largescale dynamic graph dataset comprising 1 million nodes and 7 million edges, which serves as a robust benchmark to assess the scalability and performance of our model in handling large-scale dynamic graphs. Detailed statistical information for all datasets is provided in Table 1.

5.2 Baselines

We compare our MaskDGNN with five baselines:

• EvolveGCN [Pareja et al., 2020]: It captures the evolution of GCN parameters by using sequence models like GRU or LSTM to encode parameters at each time step.
• DGNN [Manessi et al., 2020]: It employs a stacked encoder to model node dynamics by leveraging LSTM on the representations encoded by GNNs.
• dyngraph2vec [Goyal et al., 2020]: It captures temporal transitions in dynamic graphs through an auto-encoder architecture that integrates dense and recurrent layers.

1 https://github.com/heyimingheyiming/MaskDGNN

• ROLAND [You et al., 2022]: It offers a universal framework to adapt any static GNN to dynamic settings through meta-learning, by treating node embeddings as hierarchical states and recursively updating them over time.
• WinGNN [Zhu et al., 2023]: It enhances ROLAND by incorporating a sliding window mechanism to reduce model size, alleviating computational and storage demands in long-sequence scenarios.

5.3 Experiment Settings and EvaluationMetrics

In our evaluation, we use Accuracy (Acc for short), Area Under the Curve (AUC for short), Mean Reciprocal Rank (MRR for short), and Recall@10 (R@10 for short) to quantify the performance of different methods. For the baselines, we adopt the parameter settings recommended in their respective papers and further fine-tune them to achieve optimal performance. In our model, the node embedding dimension d is set to 64, the mask ratio p ranges from 10% to 40% depending on the dataset, the node dynamics score ratio α is set to 0.7, and the window size w is determined based on the snapshot length, typically ranging from 4 to 8 for each dataset. The parameters λ, η, and γ are set to 1.0, 2.0, and -0.5, respectively. The learning rate for θ, denoted by τ, is 0.008, while the learning rate for Θ, denoted by β, is set to 0.01. The dropout rate is fixed at 0.1, and the Graph Convolutional Encoder consists of 2 layers. To prevent overfitting, we apply early stopping during validation with a patience of 10. Experiments for each method are conducted ten times, and the average results are reported. All experiments are performed on four RTX 3090 GPUs with 24GB of memory.

5.4 Experiment Results

Performance Comparison (RQ1)

We evaluate the performance of MaskDGNN and all baselines in link prediction tasks on five real-world datasets. Experimental results are shown in Table 2. The best results are highlighted in bold, and the second-best ones are underlined.

As shown in Table 2, MaskDGNN outperforms all models across all metrics. For instance, on the Bitcoin-Alpha dataset, it boosts MRR by 35.86% and improves R@10, AUC, and Acc by 15.60%, 11.82%, and 6.18%, respectively. On the UCI-Message dataset, MaskDGNN surpasses the second-best model by 12.71% in MRR and 12.12% in R@10. This strong performance is due to its ability to capture intrinsic features during distribution shifts, enabling adaptation to dynamic graph changes, particularly in datasets with high node activeness and rapid fluctuations like Bitcoin-Alpha and UCIMessage. On the MOOC dataset, with higher edge density, MaskDGNN achieves 98.78% in Acc and 99.76% in AUC, benefiting from the combined effect of the node activenessaware masking and adaptive frequency enhancing strategies. Overall, MaskDGNN shows significant improvements across four key metrics, with average gains of 7.07% in Acc, 2.62% in AUC, 13.87% in MRR, and 7.31% in R@10, highlighting its robustness and effectiveness.

MaskDGNN also performs robustly on large-scale dynamic graphs. On the Wiki-Talk dataset, which contains over 1 million nodes and 7 million edges, most baseline models fail due to out-of-memory (OOM) issues. In contrast, MaskDGNN not only runs successfully but also achieves 93.30% in Acc and 99.12% in AUC. It leverages its node activeness-aware masking mechanism to remove a significant proportion of redundant edges (about 30%-40%), reducing the graph size and avoiding memory issues. Additionally, it adopts an adaptive frequency enhancing strategy, optimizing the graph data by capturing important frequency components, which enhances its ability to capture temporal features under distribution shifting.

Ablation Study (RQ2)

To analyze the effectiveness of the key components in our model, we conduct the following ablation studies:

• w/o A removes the mask rate node-activeness scores and conducts random mask mechanism.
• w/o Mask removes the self-supervised activeness-aware temporal masking module.
• w/o FE removes the frequency enhancing layer.

We evaluate the contribution of each key component in MaskDGNN by individually removing them and analyzing their impact on performance on the Bitcoin-Alpha and UCI-Message datasets. As shown in Figure 2, when the node activeness-aware temporal masking mechanism is replaced by random masking (w/o A), there is a significant performance drop, especially in Acc (from 95.54% to 91.52% in Bitcoin-Alpha, and from 87.29% to 83.40% in UCI-Message). This phenomenon highlights the crucial role of activeness-aware masking in prioritizing key temporal patterns. Similarly, the removal of the self-supervised activeness-aware temporal masking module (w/o Mask) results in an even more pronounced performance degradation, with MRR dropping from 31.53% to 14.15% in BitcoinAlpha and from 24.91% to 21.27% in UCI-Message. This further emphasizes the module's effectiveness in filtering out redundant information and focusing on key temporal changes. Additionally, removing the frequency-enhancing graph representation learner (w/o FE) reduces MRR to 23.24% in Bitcoin-Alpha and to 21.72% in UCI-Message, demonstrating the module's ability to capture intrinsic features under distribution shift. These results collectively confirm that the activeness-aware temporal masking module and the adaptive frequency-enhancing graph representation learner are key to MaskDGNN's superior performance, enabling it to effectively capture important temporal dynamics while suppressing redundant information.

Parameter Sensitivity Analysis (RQ3)

We investigate the sensitivity of four main parameters in the UCI dataset: mask ratio p, node dynamic score ratio α, embedding dimension d, and sliding window size w.

Mask ratio p. Figure 3a shows that the model achieves optimal performance at 10% masking, with stable results between 10% and 30%. Beyond 40%, performance drops sharply, indicating that while 10%-30% of edges are redundant, excessive masking may omit critical information. An appropriate mask ratio helps focus on essential features.

Node dynamics score ratio α. As shown in Figure 3b, the best performance occurs at α = 0.7, with a slight drop at α = 0.5, highlighting the importance of balancing node dynamics with global importance.

Embedding dimension d. Figure 3c shows performance improves up to d = 128, then declines. Smaller d fails to capture enough information, while larger d adds computational cost without benefit.

Sliding window size w. Figure 3d shows that MaskDGNN performs best with w = 8, effectively capturing long-term dependencies. Larger w increases cost without notable gain, so selecting an appropriate w balances performance and efficiency.

6 Conclusion

In this work, we propose a self-supervised dynamic graph neural network named MaskDGNN that combines a selfsupervised activeness-aware temporal masking mechanism with an adaptive frequency enhancing graph representation learner. MaskDGNN effectively reduces redundant information and captures intrinsic temporal features under distribution shifting. Extensive experiments on five real-world dynamic graph datasets demonstrate the superiority and effectiveness of our model.

Acknowledgments

This work is partially supported by the National Natural Science Foundation of China under Grant Nos. 62176243, 62472263, and 62402465, the Fundamental Research Funds for the Central Universities under Grant No. 202442005, and the National Key R&D Program of China under Grant No. 2022ZD0117201.