Continuous Diffusive Prediction Network for Multi-Station Weather Prediction

1 Introduction

Weather prediction plays a vital role in our life by providing crucial information for disaster preparedness [Mao et al., 2024], agricultural planning [Tabar et al., 2022], transportation safety [Zheng et al., 2023], energy management [Salehizadeh et al., 2024], and environmental protection [Bhattacharyya et al., 2022], etc. Multi-station weather prediction incorporates spatial correlations across regional stations, providing enhanced forecasting accuracy and regional flexibility, making it an important focus in modern meteorological research [Wu et al., 2023]. Traditional multi-station weather prediction typically relies on Numerical Weather Prediction (NWP) methods [Bauer et al., 2015; Yan et al., 2023; Wu et al., 2024], which use mathematical models to simulate atmospheric processes based on fundamental principles of physics and fluid dynamics. While NWP methods can produce regularly updated forecasts by solving complex equations, they require substantial computational resources and are constrained by the quality and coverage of observational data. In recent years, deep learning based weather prediction methods [Lin et al., 2022; Ma et al., 2023; Han et al., 2023; Chen et al., 2024; Bi et al., 2023; Kochkov et al., 2024] have demonstrated significant potential and advantages, due to their exceptional ability to uncover patterns from large-scale, high-dimensional, and even irregular observational data.

∗Corresponding Author

Existing deep learning based multi-station weather prediction methods can be divided into two categories, including time series prediction and spatio-temporal prediction. Timeseries prediction considers weather stations individually and focuses on analyzing the temporal dependence of meteorological variables within a single station [Zhou et al., 2021; Wu et al., 2021; Zhou et al., 2024; Wang et al., 2024b]. Zhou et al. [Zhou et al., 2022] proposed fourier enhanced blocks and wavelet enhanced blocks to capture temporal relationships through frequency domain mapping. Liu et al. [Liu et al., 2024] used fourier filtering to separate the timevariant and time-invariant components from complex nonstationary series and designed a Koopman predictor to advance respective dynamics forward. Spatio-temporal prediction considers multiple weather stations simultaneously and focuses on analyzing the spatial correlation between the stations [Lin et al., 2022; Wu et al., 2023; Xu et al., 2024; Feng et al., 2024]. Wu et al. [Wu et al., 2023] constructed a tree-based multiscale structure that considers crosscorrelation between stations. Lin et al. [Lin et al., 2022] constructed graph neural networks that consider conditional local convolutional modeling of correlation between stations. Xu et al. [Xu et al., 2024] proposed a physics-guided dynamic graph neural network, which inserts the superposition principle to capture the climate pattern.

Although previous studies achieved promising performance, they failed to accurately model the continuous physical process of atmospheric motion, leading to sub-optimal performance in weather forecasting. Since real world atmospheric motion is a complex physical process that is both spatially and temporally continuous, relying solely on the relations among discrete stations inevitably results in information loss [He et al., 2020]. Several works [Chen et al., 2018; Schirmer et al., 2022; Wang et al., 2024a; Gravina et al., 2024] also adopt the neural ordinary differential equations (ODE) methods to simulate the temporal continuity, overlooking the spatial continuity of atmospheric diffusion. In fact, modeling the spatial and temporal continuity of atmospheric processes is essential in weather prediction. Specifically, as shown in Figure 1, for one thing, the continuity could accurately characterize the physical states of observation stations at the current time step, resulting in a more precise initial physical field. For another, it enhances the model's ability to capture the physical motion of the atmosphere, thereby improving the accuracy of weather predictions for the subsequent time step. However, inferring the missing continuity is challenging without densely spaced observations.

To address the above challenges, this paper introduces the Continuous Diffusive Prediction Network (CDPNet), which models atmospheric motion continuity from both spatial and temporal perspectives. The proposed model contains a Continuous Calibrated Initialization (CCI) module and a Diffusive Difference Estimation (DDE) module, respectively representing the continuous initial physical field and the evolving atmospheric process. Specifically, the CCI module begins by interpolating data between observation stations to create a coarse spatially continuous physical field. It then refines this field by incorporating densely predicted results from the previous time step and the global directional information of the overall trend. The previous time step's predictions enhance the temporal dependence, while the directional information ensures the continuity from a global perspective. After the precise continuous representation, the DDE module explicitly models the spatial diffusion process and estimates the diffusive difference between the current time step and the subsequent one. This approach, simulating the atmospheric motion process, effectively captures motion continuity both spatially and temporally, and reflects the variations at individual stations across time steps. Similarly, we also incorporate the directional information on weather changes from the entire historical sequence to mitigate the estimation uncertainty, thereby further constraining the temporal continuity and enhancing the weather prediction performance. Extensive experiments show that the proposed model outperforms existing state-of-the-art approaches on the Weather2K and Global Wind/Temp datasets. Especially on the Weather2K dataset, the proposed model can bring at least a 4.48% improvement in Mean Absolute Error (MAE) compared to other models.

In summary, the main contributions of this paper are as follows:

• We propose the CDPNet, which models the real-world continuous weather change process from discrete station observation data, enhancing the accuracy of weather prediction.
• We propose the CCI module to construct the initial physical field through spatial interpolation and directional information calibration. It accurately represents the current physical state and provides a foundation for future weather prediction.
• We propose the DDE module to explicitly model continuous atmospheric motion through spatial diffusion process and directional information correction. It infers the evolution of the physical field and accurately predicts the weather at the next time step.
• Extensive experiments show that the proposed model significantly outperforms existing state-of-the-art models. Especially on the Weather2K dataset, the proposed CDPNet can bring at least a 4.48% improvement in MAE compared to other models.

2 Methodology

Previous studies fail to accurately model the continuous physical process of atmospheric motion, resulting in unsatisfactory weather prediction results. Therefore, we propose a weather prediction model called the Continuous Diffusive Prediction Network (CDPNet) with a Continuous Calibrated Initialization (CCI) module and a Diffusive Difference Estimation (DDE) module.

2.1 Overall Architecture

Given the historical observations X1:L = {x1, x2, · · · , xL} ∈ RN×L×C (including N weather stations, C weather variables such as temperature and wind speed over the past L time steps) and auxiliary variables Xaux (including station coordinates and time information), the goal of multi-station weather prediction is to forecast the weather conditions for the future P time steps,

{ˆxL+1, ˆxL+2, · · · , ˆxL+P} = F({x1, x2, · · · , xL}), (1) where F is the mapping function to be learned, and ˆXL+1:L+P = {ˆxL+1, ˆxL+2, · · · , ˆxL+P} ∈ RN×P×C represents the predicted future weather conditions.

We propose the CDPNet which takes into account the continuous physical process of atmospheric motion to realize multi-station weather prediction. The network structure of CDPNet is based on Recurrent Neural Networks (RNN). For each time step t, the observation xt is input to the CDPNet to predict the weather ˆxt+1 at the next time step t + 1, and this process can be expressed as ˆxt+1 = FCDPNet(xt). Specifically, CDPNet contains two key modules: the CCI module FCCI and the DDE module FDDE. The CCI module is used to construct a continuous initial physical field zt, while the DDE module models the spatial diffusion process of atmospheric motion to predict the physical field ht+1 at the next time step. After obtaining the predicted physical field at the moment t + 1, we can map the physical field back to the station weather results ˆxt+1 by a convolutional decoder D. This process can be described as

The network architecture details of CDPNet is illustrated in Figure 2.

2.2 Continuous Calibrated Initialization (CCI) Module

Constructing an accurate continuous initialization of physical fields is challenging due to the sparse distribution of weather stations across large geographic areas. Therefore, the proposed Continuous Calibration Initialization module first coarsely models the continuous physical fields through the Inverse Distance Weighted (IDW) interpolation method [Shepard, 1968], and then calibrates the coarse representation by using the predicted physical field from the previous time step as well as the weather evolution direction information from the entire observational sequence.

Specifically, to model the spatial relations among sparse weather stations, we first project these stations onto a twodimensional grid representation based on their physical locations. Therefore, each grid could indicate the physical coordinates of a weather station, and display its observational data. For locations without actual weather stations, we use the widely-used IDW interpolation method to estimate the potential weather variable values. This process yields a coarse spatial continuous initialization of physical field st ∈ RH×W×C , where H and W are hyperparameters representing the dimensions of the physical field.

Distance-based interpolation of observational data cannot capture the spatial and temporal complexity of the weather systems. Therefore, we deploy the classical Kalman filter Kt to calibrate the coarse interpolation results. The Kalman filter is traditionally used for estimating the state of a linear dynamic system from a series of noisy measurements. It typically assumes that the system dynamics and measurement noise are both Gaussian and linear, though variations of the Kalman filter are used to handle non-linear systems. In this paper, we adopt a learnable convolution to approximate the non-linear Kalman filter, and consider both the predicted physical information ht derived from the previous time step and the current direction information et extracted from the entire history sequence as the measurements. Incorporating ht and et, rt could reflect the predicted continuous physical states of the current weather stations, and thus help capture the uncertainty related to the coarse interpolation. The learnable Kalman filter determines the weight given to rt in updating the coarse interpolation results. The calibrated process can be formulated as:

where || indicates the concatenate operation, σ, tanh are the sigmoid and hyperbolic tangent activation function, and fconv is the convolutional layer. wh, we and b are learnable parameters, to incorporate the predicted physical information ht and the direction information et.

The predicted physical field ht is derived from previous observational data xt-1, encompassing both predicted spatial and temporal continuity for current observations,

Incorporating ht also helps optimize the entire weather prediction model to generate accurate weather forecasting, thereby offering an informative representation for each time step.

For the direction information et, we adopt a multi-layer perceptron fMLP to capture the direction of the weather change at each moment from the entire historical sequence X1:L to constrain the temporal continuity:

where the auxiliary features Xaux can also be combined with historical observations, such as the latitude and longitude coordinates of the station and the time at each time step. We replicate Xaux to match the dimensions of the multistation historical observations X1:L ∈ RN×L×C to obtain Xaux 1:L ∈ RN×L×C .

Through the coarse spatial interpolation and the learnable Kalman calibration, the CCI module extracts comprehensive information from the observational data and constructs a spatio-temporally continuous initial physical field, which enhances the representation of current physical states and improves future weather predictions.

2.3 Diffusive Difference Estimation (DDE) Module

Accurately modeling the temporal evolution of the continuous physical field is highly challenging due to the differences in atmospheric motion across regions. To address this, the proposed DDE module predicts atmospheric motion directions at various locations and explicitly models the continuous spatial diffusion process, thereby enabling effective forecasting of future weather changes.

The constructed initial physical field takes into account the dense prediction results of the previous time step as well as the global directional information of the overall trend. Therefore, we can estimate the future atmospheric motion from the initial physical field. Specifically, we utilize a small convolutional neural network M to estimate the displacement [ui, uj ]

Algorithm 1 CDPNet training process

• Require: Training dataset T ; 1: while not converged do 2: Sample batch of sequences {X1:L,XL+1:L+P} ∼ T 3: Computing directional information from a global perspective {eL+1, eL+2, · · · , eL+P} = fMLP (cat [X1:L;Xaux 1:L ]) 4: for t ∈ [1, L + P ] do 5: // CCI Module 6: Spatial interpolation st = IDW(xt) 7: Construct the refined physical field rt = tanh (wh ∗ ht + we ∗ et + b) 8: Kalman calibration zt = st +Kt ⊙ (rt - st) 9: // DDE Module 10: Motion estimation ut =M(zt) 11: Calculating diffusive difference ∆h = z′t - zt 12: Predict the physical field for the next time step ht+1 = zt + G (et+1) ⊙ ∆h 13: // Decoding module 14: Decoding prediction results ˆxt+1 = D(ˆht+1) 15: end for 16: Update the model parameters based on the loss function L = ∥X1:L - ˆX1:L∥ + ∥XL+1:L+P - ˆXL+1:L+P ∥ 17: end while

of the atmospheric motion at any position [pi, pj] in continuous space from the initial physical field zt. This process can be expressed as [ui, uj ] = M(zt[pi, pj ]).

Based on the estimation of atmospheric motion, the spatial diffusion process between different geographical locations can be modeled. The diffusive difference ∆h can be expressed as:

where z′t denotes the diffused physical field. We consider the impact of atmospheric motion. For each geographical location [pi, pj] in the diffused physical field z′t , it is obtained by diffusing the initial physical field features zt from the location [pi -ui, pj -uj ]. The atmospheric motion displacement ui, uj are obtained from the motion estimation network M described above.

However, the diffusion process of a single time step is difficult to ensure the continuity of the weather change from a global perspective. Therefore, we consider inferring the direction information of the weather change at the current moment from the whole historical weather series to correct the diffusion difference. The complete diffusive difference estimation process can be written in the following form:

where ht+1 is the physical field at time step t + 1 estimated from the initial physical field zt. The G (·) is a simple convolutional network, and the direction information et+1 is computed in the same way as in Eq. (6).

Therefore, the DDE module considers the atmospheric motion and explicitly simulates the spatial diffusion process, while the weather change direction information extracted based on the whole historical sequence is used to constrain the time continuity and improve the weather prediction performance.

papers, the data for training, validation, and testing are all one-year time. The task is set to predict 12 hours in the future based on the past 12 hours, where the input length is 12 steps and the prediction length is 12 steps.

Global Wind/Temp: It is from the National Centers for Environmental Information. This dataset contains the hourly averaged wind speed and hourly temperature of 3,850 stations around the world from January 1, 2019 to December 31, 2020. Like the setup in other papers, we split the dataset into training, validation, and test sets in chronological order by a ratio of 7:1:2. The task is set to predict one day in the future based on the past 2 days, where the input length is 48 steps and the prediction length is 24 steps.

Implementation details

Our model is implemented using PyTorch 2.1.0 and trained on an NVIDIA GeForce RTX 2080 Ti GPU. We employ the Adam optimizer with a batch size of 1 to accommodate the large number of stations in our dataset. The training process consists of two distinct phases: first, we initialize the direction information by training for one epoch with a learning rate of 0.001, followed by training the entire network with a reduced learning rate of 0.00001. The model undergoes training for up to 100 epochs, with an early stopping mechanism implemented to prevent overfitting. The implementation code is publicly available at https://github.com/ChujieXu/CDPNet.

Evaluation metrics

We measure our model and other methods by three common deviation-based evaluation metrics: Mean Squared Error (MSE), Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE).

2.4 Optimization

The overall training flow is shown in Algorithm 1, and we use an iterative generation approach to predict the weather at the next time step based on the information at the current time step, which is consistent with the RNN architecture. Specifically, for historical moments, we reconstruct the entire historical sequence X1:L, this is to ensure that the evolution of the weather field is consistent with the variation of the real weather. For future moments, the model generates the final result ˆXL+1:L+P .

The optimization objective during training is two-fold: to minimize the reconstruction error of the historical sequence and the prediction error of the future sequence. This is formalized as follows:

where both parts of the loss are optimized using l1 loss.

3 Experiments

In this section, extensive experiments were conducted to validate the effectiveness of the proposed CDPNet.

3.1 Experimental Settings

Data description

In this paper, experiments are carried out on two real datasets, including the Weather2K dataset [Zhu et al., 2023] and the Global Wind/Temp dataset [Wu et al., 2023].

Weather2K: It covers 31 provinces in China and contains 1,866 ground weather stations. The time scale is from January 1, 2017 to December 31, 2019. Based on meteorological consideration, 20 important near-surface meteorological factors and 3 time-invariant constants for position information are provided in the Weather2K. Like the setup in other

3.2 Performance Comparison

To comprehensively demonstrate the effectiveness of the proposed CDPNet model, we compared it with various prediction models based on different motivations, which can be roughly classified into two categories: time series prediction models, such as Autoformer [Wu et al., 2021] and FEDformer [Zhou et al., 2022], focusing on the time dependence of meteorological variables at individual stations; and spatiotemporal predictive models, such as ASTGCN [Guo et al., 2019], MSTGCN [Guo et al., 2019], DCRNN [Li et al., 2018], GCGRU [Seo et al., 2018], Corrformer [Wu et al., 2023] and EasyST [Tang et al., 2024], which focus on spatial correlations between multiple stations. Additionally, we considered other methods incorporating physical information, such as PhyDNet [Guen and Thome, 2020] and NowcastNet [Zhang et al., 2023]. Based on the proposed CDPNet, we compared the key physical modules of these methods with our proposed DDE module.

As can be seen from Table 1, our model outperforms other comparison methods using both MAE and RMSE metrics on the Weather2K dataset, which indicates the superiority of CDPNet. For the four most common meteorological variables of air temperature, relative humidity, wind speed, and air pressure, CDPNet obtained an average MAE of 2.7372 and an average RMSE of 3.7045, which are 4.48% and 3.87% improvement compared to the suboptimal method, respectively. As shown in Table 2, we similarly validated on the Global Wind/Temp dataset with a wider range and a larger number of stations. From Table 2, we can see the performance gain is also evident, demonstrating the practicability of our model. For the two variables of temperature and wind speed, CDPNet obtained an average MSE of 5.244 and an average MAE of 1.562, which obtained the best performance.

Besides the overall performance evaluation, we also compared the prediction errors across different forecasting lengths, as shown in Figure 3. The prediction errors of the proposed CDPNet consistently remain lower than that of NowcastNet. On the other hand, Corrformer achieves higher accuracy in the short term but experiences a sharp decline in performance as the forecasting length increases. This indicates that the proposed CDPNet is better at capturing longtermweather trends.

3.3 Ablation Study

In this section, we conduct ablation studies to analyze the effectiveness of the designed components in the proposed model. Table 3 outlines the findings of the baseline model, baseline model with the proposed CCI module, baseline model with the proposed DDE module, and baseline model with CCI module and DDE module (our CDPNet). The baseline model with CCI module means that this approach does not perform complex diffusive difference estimation. Instead, it uses a simple convolutional network to map the constructed initial physical field to future time steps for prediction. The baseline model with DDE module means that instead of performing continuous calibrated initialization, it employs simple reshaping and convolution operations to transform discrete stations into a two-dimensional space. As shown in Table 3, CCI module helps real-world modeling to predict weather conditions due to its ability to construct a continuous initial physical field. The MAE decreases from 5.4633 to 2.9540, and the RMSE decreases from 6.9528 to 3.9638. After adding the DDE module to estimate the atmospheric motion considering the weather evolution, a more significant performance improvement is observed. The MAE of the proposed model decreases from 5.4633 to 2.8442 and the RMSE decreases from 6.9528 to 3.8228. When both CCI module and DDE module are applied to the baseline model, the model structure is the same as the proposed CDPNet model. The model achieves an optimal MAE of 2.7372, as well as an optimal RMSE of 3.7045. All these results indicate that each component of our model contributes significantly to the final CDPNet model, resulting in the best performance.

3.4 Parameter Sensitivity

The core of the proposed CDPNet method lies in the initialization and diffusion of the physical field. Therefore, we conducted experiments to explore the impact of the physical field size on prediction performance. As shown in Figure 4, we compared the effects of the physical field sizes of 48×48, 64×64, and 80×80 on the final prediction performance. The experiments revealed that the size of the physical field had little overall impact on the results. This is because the physical field features already exhibit strong spatio-temporal continuity, and changes in resolution have a limited effect on extracting key information. Even at lower resolutions, the primary physical trends can still be captured effectively.

3.5 Visual Analysis

To validate the effectiveness of the proposed CDPNet method, we visualize the prediction results in a local area of northern China. As shown in Figure 5, a comparison is presented between the temperature predictions of CDPNet (our model) and Corrformer (comparison method) with the ground truth for the next 3, 6, 9, and 12 hours. Each row represents a different model, and each column corresponds to the predictions at different times. Redder colors indicate higher temperatures, with the color bar on the right displaying the temperature range. Additionally, there are error maps below the predictions of CDPNet and Corrformer, where color intensity reflects the prediction error, allowing for a visual comparison of each model's accuracy at different times. The figure reveals that CDPNet has smaller errors in temperature predictions for the next 3, 6, 9, and 12 hours, with wider low-error regions. In contrast, Corrformer has larger errors, particularly at 9 and 12 hours. This indicates that CDPNet, through continuous modeling, effectively captures the spatiotemporal dependencies of temperature variations, enabling it to more accurately reflect the dynamic temperature changes, thus achieving higher prediction accuracy across different periods.

4 Conclusion

In this paper, the CDPNet is proposed to improve the accuracy of multi-station weather prediction by capturing the continuity of atmospheric motions in both space and time. CDPNet simulates the initial physical field and the atmospheric diffusion process of the weather station through the CCI module and the DDE module. Experimental results show that CDPNet can effectively capture the continuity of atmospheric motion in space and time, and outperforms previous deep learning methods on multiple datasets. In particular, on the Weather2K dataset, CDPNet demonstrates a significant advantage in terms of MAE.

Acknowledgments

This work was supported by National Natural Science Foundation of China under Grant 62476018.