DGraFormer: Dynamic Graph Learning Guided Multi-Scale Transformer for Multivariate Time Series Forecasting

1 Introduction

In the context of the digital era, time series forecasting has become a crucial focus for businesses and organizations, offering significant applications across various domains, including finance [Cao, 2022], energy [Zhu et al., 2023], meteorology [Zhao et al., 2024], transportation [Cao et al., 2025]. These applications often involve vast amounts of multivariate time series data. Multivariate time series refers to multiple sequences of data evolving over time, where each sequence represents a specific time-dependent variable. These sequences often show inherent interdependencies or correlations. By analyzing and modeling historical data, organizations can predict trends, detect anomalies, and facilitate datadriven decision-making, ultimately optimizing economic activities and resource allocation.

∗Corresponding author: Yanwei Yu.

Deep learning models like Recurrent Neural Networks [Chen et al., 2021] and Long Short-Term Memory networks [Zhao et al., 2017] excel at processing nonlinear, nonstationary time series data with time lags and seasonality, which has promoted the development of time series forecasting. Recently, Transformer achieves outstanding success across various fields [Karita et al., 2019; Brown et al., 2020; Chang et al., 2024], and it shows great promise in time series forecasting as well [Chen et al., 2024; Zhou et al., 2024b].

Although a few transformer-based methods have been proposed for multivariate time series forecasting, they have neglected the explicit modeling of correlations between variables to better explore their complex relationships. For instance, PatchTST [Nie et al., 2023] adopts a channel independence approach, treating each variable separately and intentionally ignoring the complex correlations between them. Thus, a key challenge in advancing multivariate time series forecasting is how to capture these inter-variable correlations better. Graph Neural Networks offer an effective solution by representing multivariate time series as graphs, where variables are nodes and edge weights indicate inter-variable correlations [Chen et al., 2023]. Multivariate time series forecasting faces a challenge for obtaining a good dynamic graph representation due to the dynamic and complex relationships among variables. Generally, existing methods have made some progress in correlation modeling, but two issues remain.

Dynamic nature of multivariate correlations: The relationships between variables are not static, and they can evolve over time, showing periodic or trend-based changes. As shown in Fig. 1, by treating time points as a time window, we observe that the correlation similarities vary across time windows. By learning from real sequence data, we can capture the dynamic relationships that evolve across different windows. Existing method (e.g., MTGNN [Wu et al., 2020]) has used a single graph to propagate correlation information across the entire time scale, and it fails to capture the varying correlation weights across different time windows. This limits the accuracy of feature aggregation based on overall sequence correlation, thereby affecting prediction precision.

Noise caused by negative relationships: Although existing methods (e.g., MSGNet [Cai et al., 2024]) have explored learnable graph structures, they often struggle to learn the optimal structure. This difficulty arises because realworld datasets are typically influenced by various unforeseen factors, which can cause uncorrelated variables to exhibit spurious similarities. As a result, these variables might propagate negative relationships (noise information) during the message-passing process, leading to model performance degradation. The noise can hinder the effective transmission of critical information between key variables, ultimately reducing the ability of model for capturing essential patterns.

To address the challenges outlined above, we propose a Dynamic Graph Learning Guided Multi-Scale Transformer (DGraFormer), which comprises two main components: Dynamic Correlation-aware Graph Learning (DCGL) and MultiScale Temporal Transformer (MTT). The method views time series data as multiple time windows with stable correlations. It adaptively learns the graph adjacency matrices for each time window, filters out noise, and selects the most crucial correlation weights to guide the aggregation of feature representations based on correlations. This approach significantly improves the ability of the model to adapt to dynamic correlation changes and overcomes the issue of redundant information propagation caused by external noise in existing methods. Our contributions are summarized in three main aspects:

• We propose a novel multivariate time series forecasting model, named DGraFormer, which combines DCGL and MTT in a unified framework to simultaneously achieve the dynamic correlation-aware graph learning and multi-scale temporal information interaction.
• A DCGL is designed to adaptively capture the dynamic correlations among multivariate variables in temporal windows, and then select essential information to guide the process of correlation feature aggregation, effectively eliminating noise in the weights and learning critical corresponding adjacency weights for dynamic graph learning.
• Extensive experiments on ten real-world datasets demonstrate that DGraFormer outperforms SOTA multivariate time series forecasting models. Compared to the latest graph-based model, MSGNet, our model achieves average reductions of 11% in MSE and 10.6% in MAE.

2 RelatedWork

2.1 Time Series Forecasting

Time series forecasting has been extensively studied. Recently, Transformer [Vaswani, 2017; Kitaev et al., 2020; Tang and Matteson, 2021] shows superior performance in semantic extraction and long-range dependency modeling, outperforming methods based on TCNs [Bai et al., 2018; Liu et al., 2022a; Lai et al., 2023] and RNNs [Rangapuram et al., 2018; Qin et al., 2017]. This trend drives interest in Transformer-based models. Informer [Zhou et al., 2021] has led the adoption of Transformer for time series forecasting. Autoformer [Wu et al., 2021] introduces seasonal trend decomposition, while FEDformer [Zhou et al., 2022] extends this approach by transforming the time domain into the frequency domain for decomposition learning. Pyraformer [Liu et al., 2022b] employs convolution to capture multi-scale features, and Non-stationary Transformers [Liu et al., 2022e] focus on time series stationarization.

Despite challenges raised by linear models [Zeng et al., 2023; Das et al., 2023], Transformers remain top-performing in time series forecasting [Jiang et al., 2023]. PatchTST [Nie et al., 2023] leverages channel independence by representing each variable as a separate channel and segmenting it into subsequence-level patches, which are subsequently used as tokens for the Transformer model. In contrast, Crossformer [Zhang and Yan, 2023] adopts variable patching and a hierarchical encoder-decoder to capture cross-dimensional dependencies. Recent work [Zhou et al., 2024a] proposes inputting the same number of auxiliary variables for each variable into the Transformer for learning. And iTransformer [Liu et al., 2024] encodes variables independently as distinct tokens, using attention for inter-variable and feedforward networks for temporal dependencies. Further techniques are under exploration [Pan et al., 2021; Deng et al., 2022; Li et al., 2023; Liu et al., 2023; Deng et al., 2024a; Deng et al., 2024b], including TimesNet [Wu et al., 2023], which projects one-dimensional time series data into a twodimensional space and analyzes it with sophisticated visual networks.

Most existing methods overlook the modeling of correlations between variables. Existing methods for learning variable correlations fail to account for the dynamic nature of multivariate correlations and the noise in correlation modeling. These approaches propagate information across all variables, while neglecting isolated variables (those not correlated with others).

2.2 GraphModeling for Multivariate Time Series Forecasting

The use of GNNs [Kipf and Welling, 2016; Defferrard et al., 2016; Abu-El-Haija et al., 2019] introduces a new perspective to model multivariate time series data. However, most existing approaches apply them to specific domains, such as traffic [Yu et al., 2017; Li et al., 2017; Wu et al., 2019; Wen et al., 2023; Liu et al., 2022c], where prior knowledge (e.g., Euclidean distance or POI) can naturally define graph structures. In contrast, general multivariate time series forecasting models involve datasets from multiple domains, making it difficult to obtain a unified, predefined graph structure.

To address this, some methods [Wu et al., 2020] have explored learnable graph structures, replacing predefined graphs with learnable parameter matrices. However, methods such as MAGNN [Chen et al., 2023] and MSGNet [Cai et al., 2024] still employ static graphs to represent time series, which are insufficient to capture the dynamic nature of variable correlations. Although TPGNN [Liu et al., 2022d] captures temporal dynamics through matrix polynomials, it ignores noise caused by external factors. Consequently, effectively learning the core correlations between variables remains a critical challenge in correlation modeling.

3 ProblemDefinition

For multivariate time series forecasting, we denote the input data over a look-back window as X = {x1, . . . , xT } ∈ RN×T , where T is the number of time steps and N is the number of variables. The objective is to forecast the next τ time steps, represented as Y = {xT+1, . . . , xT+τ } ∈ RN×τ .

We model the multivariate time series data using graphs G = {V, E}, where V denotes the set of nodes and E ∈ RN×N is the correlation weight matrix between the nodes. To capture the dynamic correlations between variables, we divide the time series data into W windows, each containing m time steps. The correlation graph structure is then represented as G = {V, {E1, E2, . . . , EW}}, where Ew ∈ RN×N is the correlation weight matrix for the w-th time window.

4 Methodology

To effectively capture multivariate correlations, we propose a novel multivariate time series forecasting model called DGraFormer. The overall architecture of DGraFormer is illustrated in Fig. 2, comprising two main components: Dynamic correlation-aware graph Learning (DCGL) and MultiScale Temporal Transformer (MTT).

First, the DCGL generates the corresponding graph structure matrices with dynamic essential multivariate correlations for the input sequence. Then it takes the instance-normalized data and its corresponding edge weight matrices to learn the correlation representations. The MTT extracts temporal representations from the patch data at varying scales. Finally, the learned representations, which encapsulate both correlation and temporal information, are passed through the Flatten and Linear layers in the predictor to produce the final forecast output.

4.1 Dynamic Correlation-aware Graph Learning

The Dynamic Correlation-aware Graph Learning is designed to capture graph structures that reflect dynamic correlations and focus on essential correlation weights. We construct the weight matrix A ∈ RN×N×T , which guides the propagation of the correlation information for the input data X.

Dynamic Multivariate Correlation Weight Learning

Existing methods have modeled the entire time series with a single graph, which failed to capture dynamic changes in multivariate correlations. To overcome this limitation, and enable more precise correlation modeling, we divide the time series into W time windows, each containing m data points. For each time window, we independently learn its correlation graph weights.

First, we compute the overall correlation weights based on the information from the entire dataset used for training. It represents the general relationships between variables in the training set Xall ∈ RN×M , where M denotes the total number of time steps in the training set. Raw data is often influenced by trends and external factors, which complicates the extraction of the underlying correlations among variables masked by the overall trend. To mitigate these influences, we transform the data from the time domain to the frequency domain, removing trend components and extracting seasonal information. Specifically, we decompose Xall into Fourier bases using the Discrete Fourier Transform (DFT(·)), select the top Kf amplitudes, and apply the Inverse Discrete Fourier Transform (IDFT(·)) to obtain the seasonal representation of the data, denoted as Xsea ∈ RN×M :

where A is the set of amplitudes for all selected frequency components, and Φ is the set of phase angles for all selected frequency components. Using Xsea, we calculate the cosine similarity between all pairs of variables, which results in the overall matrix of correlation weights C ∈ RN×N :

Secondly, we generate a unique parametric correlation matrix for each time window and optimize it during the learning process. This enables precise modeling of variable correlations by capturing the fine-grained correlation graph structures within different time windows. Specifically, for the matrix of weights Rw ∈ RN×N of the w-th time window, it is computed as follows:

where (Ew)1 and (Ew)2 ∈ RN×h denote randomly initialized node embeddings (with each row corresponding to a variable). The subscripts 1 and 2 indicate different initialization instances, both of which are learnable during training. h is the embedding dimensionality.

Finally, we aggregate C and Rw to derive the correlation graph weights matrix Ew ∈ RN×N for the w-th time window:

where α governs the relative contribution of C to Ew. Initially, α is set to 0.9 and is gradually decreased to 0.1 as training progresses. This adjustment ensures that the parameterized weights matrix Rw progressively takes precedence over C in Ew, gradually reflecting the unique correlations among variables in the w-th time window.

Essential Correlation Information Focusing

To eliminate redundant information that may interfere with the learning of correlation features, such as temporary false correlations induced by external factors, and to focus on the essential correlation weights, we focus essential correlation information to Ew for each time window. Specifically, we generate a corresponding mask matrix Mw ∈ RN×N for the w-th time window by selecting the top Ke values from the vectorized form of Ew:

Given the learnable nature of Ew, Mw evolves dynamically during training, adaptively guiding the model in selecting the essential correlation weights. We apply the mask matrix to Ew as follows Ew :

where ˜ Ew ∈ RN×N denotes the sparsified correlation weight matrix for the w-th time window, and ⊙ indicates elementwise multiplication. We apply the above operation to the correlation edge weight matrix of each time window, obtaining a set of dynamically sparsified correlation edge weight matrices ˜ E = {˜ E1, ˜ E2, . . . , ˜ EW } ∈ RN×N×W .

For each input data X ∈ RN×T , we apply advanced indexing to extract the weight matrix A ∈ RN×N×T for correlation feature learning:

where I ∈ RT×1 denotes the index information of the time window for each time point. We then feed A into the GCNbased correlation fusion module to guide the information propagation process.

Correlation-aware Graph Message Passing

To propagate information based on the output weight matrix A, we use input data processed with Instance Norm, which is a technique used to mitigate distribution shifts between training and testing data, to learn feature representations that aggregate essential correlations. Inspired by MTGNN [Wu et al., 2020], we adopt a graph convolution approach that incorporates residual connections and the mix-hop concept for learning correlation representations. The information propagation step is defined as follows:

where H(0) denotes the input data X processed with Instance Normalization and feature dimensional upsampling, and H(l) represents the result of the l-th layer of message passing. β controls the proportion of the root node's original state retained, ensuring that the propagated node states explore the depth of the neighborhood while preserving locality. We use the following approach for information selection:

where the learnable parameter matrix W acts as a feature selector, and LMP represents the total number of layers for message passing. It adaptively selects the message passing features at different depths to generate the output Hout ∈ RN×T of the correlation fusion module.

4.2 Multi-Scale Temporal Transformer

The Multi-Scale Temporal Transformer is designed to capture the temporal patterns within sequence data. Existing methods, such as PatchTST [Nie et al., 2023], have shown that dividing time series into patches and using them as tokens in a Transformer Encoder for feature learning can significantly improve the accuracy of multivariate time series prediction. Aggregating time steps into subsequence-level patches enhances locality and captures comprehensive semantic information that cannot be obtained at the point level. However, using a single patch size to divide the time series overlooks the features present at different time resolutions. To address this, we adopt a multi-scale patch division and combination strategy to model temporal correlations.

Patch Division

For the output Hout ∈ RN×T of the correlation fusion module, we divide the i-th univariate time series hiout ∈ R1×T into S = T/P non-overlapping patches, where P represents the number of time steps in each patch, denoted as hip ∈ RP×S . We then apply a trainable linear projectionWp ∈ RD×P to map the patches to the Transformer latent space of dimension D. In addition, we use a learnable additive position encoding Wpos ∈ RD×S to capture the temporal order of the patches:

where ¯xi ∈ RD×S denotes the univariate input to the Transformer encoder.

Transformer Encoder

The structure of our transformer encoder follows the standard vanilla Transformer architecture. Initially, the multihead self-attention mechanism is used to model the temporal correlations between different patches. By applying a linear transformation to the input ¯xi , we obtain the query, key, and value matrices, denoted as Qi ,Ki ,Vi . The attention output zia ∈ RD×S is then computed as follows:

Subsequently, the attention output is processed through the Add & Norm layer, which integrates residual connections and batch normalization to improve training stability and convergence. This operation can be mathematically expressed as:

where Norm(·) denotes the BatchNorm. Following the Add & Norm layer, the intermediate representation is further refined using a position-wise feed-forward network (FFN). The FFN is composed of two linear transformations separated by a non-linear activation function. The output of this stage is given by:

As a result, the transformer encoder produces the multivariate output Z ∈ RN×D×S , which encapsulates the temporal feature representation of the input sequence.

Patch Combination and Multi-Scale Encoding

To capture the diverse features embedded in multi-scale patches, we combine neighboring patches pairwise and input them into another identical Transformer encoder for further learning. This process is represented as:

where Z(l) ∈ RN×D(l)×S(l) represents the output of the lth layer of the transformer encoder TE(l) (·), which includes multi-head attention, BatchNorm layers, and feed-forward network. The input to the first encoder TE0 (·) is denoted as X(0) p , and its output as Z(0) . Through multiple iterations, we learn feature representations that capture information across different time resolutions. Finally, After being processed by LTE layers of Transformer encoder, the output of the last Transformer encoder is flattened, and a Linear layer is applied to generate the final prediction result Y = {xT+1, . . . , xT+τ } ∈ RN×τ .

5 Experiments

The experiments are designed to address the following research questions:

• RQ1: What is the performance of our DGraFormer as compared to various state-of-the-art baselines?
• RQ2: How do the key components contribute to the performance?
• RQ3: How do the key hyperparameters influence the performance of the proposed DGraFormer?

5.1 Datasets

In our experimental evaluation, we employ 10 real-world datasets, including ETT (4 subsets), Weather, Electricity, Solar-Energy, Traffic used by iTransformer [Liu et al., 2024], Flight evaluated in MSGNet [Cai et al., 2024] and AirQualityUCI 1 , to assess the performance of all methods.

5.2 Baselines

We compare our DGraFormer against with four categories of baselines: (1) Graph-based method: MSGNet [Cai et al., 2024]; (2) Transformer-based methods: iTransformer [Liu et al., 2024], PatchTST [Nie et al., 2023], Crossformer [Zhang and Yan, 2023], Pyraformer [Liu et al., 2022b]; (3) Linearbased methods: DLinear [Zeng et al., 2023], TiDE [Das et al., 2023]; (4) TCN-based method: TimesNet [Wu et al., 2023].

5.3 Experiment Settings and EvaluationMetrics

DGraFormer uses the Adam optimizer and a combination of L1 and L2 loss for model training. The number LMP of graph message passing layers is set to 2, and the number LTE of the Transformer encoder is set to 3. Additionally, a patch length p = 8 is used for division. The number of time points m in each time window depends on the dataset's time frequency, corresponding to the number of data points in one day (e.g., 24 for Electricity, 144 for Solar). The weight focusing ratio Ke ranges from 0.5 to 0.005 based on the number of variables in the dataset, decreasing as the number of variables increases. And we set Kf=3 in Eq. (1). All experiments are conducted using PyTorch and executed on an NVIDIA GeForce RTX 4090 24GB GPU. To ensure fair comparisons, all models are configured with the same look-back window T = 96 and prediction lengths τ ∈ {96, 192, 336, 720} for all datasets. The hyperparameters for the baseline models are set according to the configurations provided in the original papers. We evaluate model performance using two common metrics in multivariate time series forecasting: Mean Absolute Error (MAE) and Mean Squared Error (MSE).

1 https://github.com/Gauhar1107/AirQualityUCI

5.4 Experiment Results

Performance Comparison (RQ1)

The results of multivariate time series forecasting are presented in Table 1, DGraFormer achieves optimal performance on most of the evaluation metrics. Compared with 2024 SOTA models iTransformer and MSGNet, DGraFormer reduces MSE and MAE by an average of 6.75% and 7.25%. All models face performance decline with longer prediction lengths due to information decay and error accumulation. Yet DGraFormer consistently outperforms SOTA, e.g., exceeding MSGNet by 10.5%/10.8% (MSE/MAE) at horizon=192, 11.7%/11.0% at 336, and 10.5%/9.8% at 720. This demonstrates its robust capability in long-term forecasting.

Specifically, on high-dimensional datasets (Electricity, Solar, Traffic) with 100+ variables, DGraFormer reduces MAE by 6.6% over iTransformer by (1) capturing richer intervariable correlations via graph attention and (2) leveraging more contextual information for improved accuracy. Although Transformer-based models (e.g., iTransformer) employ multi-head attention to capture inter-variable correlations, they do not explicitly model multivariate correlations. In contrast, graph-based methods more flexibly model dynamic, complex variable correlations. iTransformer uses shared attention across variables, limiting its ability to capture evolving, heterogeneous correlations in multivariate series.

Despite both using graph-based correlation modeling, DGraFormer outperforms MSGNet by 11% MSE and 10.6% MAE on average. On MSGNet's FLight dataset, DGraFormer reduces MSE by 24% and MAE by 18.7%. While MSGNet learns graph structures across multiple scales, it does not dynamically model within each scale or handle noise effectively. This demonstrates that dynamic modeling and focusing on key correlations can significantly improve forecast accuracy.

Compared with linear-based models TiDE and DLinear, DGraFormer reduces MSE/MAE by 21.7%/17.9%. This further confirms that a well-designed Transformer model can still achieve strong performance in time series forecasting.

Ablation Study (RQ2)

We ablate DGraFormer's components with four variants:

• w/o Dynamic Time Windows (DTW): Using a single graph to learn variable correlations across all time steps.
• w/o Dynamic Graph Learning (DGL): Using only the initial adjacency matrix C to model variable correlations.
• w/o Essential Correlation Weight Focusing (ECF): Using all positive weights for graph message propagation.
• w/o Multi-scale Transformer Encoding (MTE): Using a fixed uniform patch size across all Transformer encoder layers for temporal learning, without patch combination.

Fig. 3 illustrates the predictive performance of different model variants on Solar, Electricity, and AirQualityUCI datasets. Removing the dynamic time window caused a substantial decrease in accuracy, highlighting the importance of modeling dynamic correlations in inter-variable relationships. The removal of dynamic graph learning impairs the model's ability to adaptively capture correlations among variables. Excluding the essential correlation weight focusing resulted in the model's inability to effectively filter out noise, which disrupted the communication of core correlations. Finally, removing multi-scale encoding restricted the model's capacity to capture time features at different scales. Overall, graph learning effectively captures the correlations between variables, while the Transformer excels in learning temporal features across different time scales. The combination of these components results in improved prediction performance.

Parameter Sensitivity Analysis (RQ3)

We analyze the parameter sensitivity of DGraFormer with respect to three key hyperparameters, the number m of time points, the weight focusing ratio Ke, and the cosine similarity matrix proportion α on Weather and Solar datasets.

Fig. 4a and 4b show the prediction performance for varying values of m, the number of time points in each time window. The model performs best when m = 144, corresponding to a full day, as correlations within a single day may tend to be stable. Larger time windows, however, introduce varying correlations, reducing performance.

Fig. 4c and 4d assess the impact of the weight focusing ratio Ke. As Ke decreases, performance improves up to a point before declining. A moderate decrease helps eliminate redundant noise, preserving essential correlations for accurate feature aggregation. However, too low a Ke discards important weights, weakening model performance.

Fig. 4e and 4f assess the cosine similarity matrix proportion α. Performance remains stable across α values, as graph structure learning adaptively adjusts weights to mitigate fluctuations. The cosine similarity matrix helps prevent random initialization fitting issues and guides the model to adjust based on a correlation matrix that reflects seasonal patterns. Graph learning compensates for changes introduced by varying retention ratios.

6 Conclusion

In this paper, we propose the Dynamic Graph Learning Guided Multi-Scale Transformer (DGraFormer) to address challenges in multivariate time series forecasting. The model integrates two key components: Dynamic Correlation-aware Graph Learning (DCGL), which captures dynamic correlations and filters noise, and Multi-Scale Temporal Feature Learning (MTF), which extracts temporal patterns at multiple scales. Experimental results demonstrate that DGraFormer outperforms sota methods on ten real-world datasets, achieving superior performance.

Acknowledgments

This work is partially supported by the National Natural Science Foundation of China under Grant Nos. 62176243, 62372421, and 62402463, the Fundamental Research Funds for the Central Universities under Grant No. 202442005, and the National Key R&D Program of China under Grant No. 2022ZD0117201.