Physics-Assisted and Topology-Informed Deep Learning for Weather Prediction

1 Introduction

Weather prediction is of paramount importance to social security and economic development, and has attracted extensive research efforts since the ancient time. Among the modern weather prediction methods, numerical weather prediction (NWP) is built upon differential equations that govern the weather evolution [Randall et al., 2007; Bauer et al., 2015]. These differential equations attribute the weather evolution to the advection process and the Earth-atmosphere interaction [Rood, 1987; Smith et al., 1990], as shown in Figure 1. The advection process is the evolution of weather variables (described by the advection equation) driven by the evolution of their velocity fields (described by the NavierStokes equation) [Temam, 1984]. The Earth-atmosphere interaction encompasses other complex physical processes in the atmosphere, such as radiation, clouds, and subgrid turbulent motions. One particular challenge in NWP is that the Earth-atmosphere interaction is difficult to model and calculate, forming a bottleneck of improving the accuracy of NWP [Hourdin et al., 2017; Kochkov et al., 2024]. Besides, the accuracy of NWP does not improve with the increasing amount of historical observations.

On the other hand, data-driven methods that predict the weather based on the historical observations, especially deep learning models, have become very popular in recent years [Bouall`egue et al., 2024]. With the aid of high-quality and ever-accumulating data, state-of-the-art deep learning models have demonstrated great potentials and been integrated into the modern weather prediction systems [Kurth et al., 2023; Bi et al., 2023; Lam et al., 2023]. In addition, deep learningbased models are able to remarkably shorten the time consumption in the prediction stage [Kurth et al., 2023]. However, these models disregard either the physics of the weather evolution or the topology of the Earth's surface. Thus, their predictions are often unreliable due to the lack of the physical constraints or suffer from the distortions caused by the topological structure [Schultz et al., 2021; Xu et al., 2024a].

1.1 Enhancing Deep Learning with Physics

Combining with the differential equations that characterize the weather evolution can enhance the precisions, efficiency and robustness of deep learning models, because the differential equations provide valuable prior knowledge [Xiang et al., 2022]. Some works incorporate differential equations into losses during training deep learning models [Daw et al., 2021]. Nevertheless, tuning weights for the differential equations and computing stochastic gradients of the losses bring new challenges. Some other works use deep learning models to correct NWP models [Kwa et al., 2023; Xu et al., 2024b; Kochkov et al., 2024]. Though having high accuracies, these approaches are computationally demanding since they need to both solve a large system of differential equations and train end-to-end neural networks. The closest to ours are [Zhang et al., 2023; Verma et al., 2024], in which neural networks are trained with the aid of differential equations. However, they both overlook the Navier-Stokes equation that drives the evolution of the velocity fields.

Despite that these physics-assisted deep learning models are harder to train and slower in inference compared to the end-to-end deep learning methods, they significantly enhance the robustness of predictions and demonstrate remarkable potentials [Chen et al., 2018].

1.2 Taking Topology of Earth into Consideration

The historical observations used during training most deep learning-based weather prediction models are often on planar latitude-longitude grids, other than on the spherical surface of the Earth. However, neglecting the topology of the Earth's surface introduces remarkable distortions, as shown in Figure 2 [Cohen et al., 2018; Mai et al., 2023]. For example, the points that are close to the poles turn to be denser on the spherical manifold than on the planar latitude-longitude grid. A notable consequence is that one weather pattern appears differently on the sphere and the plane, such that capturing the weather pattern on the plan suffers from distortions. These distortions also affect the patches and convolution kernels, negatively impacting the deep learning models based on convolutional neural networks or transformers [Coors et al., 2018]. In addition, the velocity fields defined on the planar are significantly distorted when increasing the latitude towards the poles, which will bring biases to the deep learning models that learn the velocity fields [Zhang et al., 2023; Verma et al., 2024].

1.3 Contributions

In this paper, we propose PASSAT, a novel Physics-ASSisted And Topology-informed deep learning model for weather prediction. PASSAT attributes the weather evolution to the analytical advection process and the complex Earth-atmosphere interaction. In the advection process, the evolution of weather variables is driven by the evolution of their velocity fields, and the two are respectively described by the advection equation and the Navier-Stokes equation. PASSAT also takes the topology of the Earth's surface into consideration. Therefore, PASSAT: (i) trains a spherical graph neural network to estimate the Earth-atmosphere interaction; (ii) generates the initial velocity fields with the same spherical graph neural network; (iii) numerically solves the advection equation on the spherical manifold; (iv) updates the velocity fields through numerically solving the Navier-Stokes equation on the spherical manifold. Our contributions are as follows.

• PASSAT seamlessly integrates the historical observations, the physics of the weather evolution and the topology of the Earth's surface, yielding a novel physicsassisted and topology-informed deep learning model for weather prediction.
• Compared to the black-box deep learning models, PASSAT takes advantages of the physical constraints, characterized by the advection equation and the NavierStokes equation, and thus remarkably improves the quality of medium-term prediction.
• Compared to the NWP models, PASSAT avoids modeling and calculating the complex Earth-atmosphere interaction. PASSAT is also able to utilize the historical observations to improve the prediction accuracy.
• PASSAT solves the differential equations and trains the graph neural network on the spherical manifold other than on the planar latitude-longitude grid, and thus effectively avoids the distortions brought by the latter.
• We conduct experiments on the 5.625◦ ERA5 data set, demonstrating the competitive performance of PASSAT compared to the state-of-the-art deep learning models and the NWP model IFS T42.

2 RelatedWorks

Numerical weather prediction (NWP). NWP is a fundamental physics-based method for weather prediction [Scher, 2018], utilizing the underlying differential equations to predict how the weather should evolve over the time. For example, the operational Integrated Forecast System (IFS) consists of several NWP models with different spatial resolutions [Bouall`egue et al., 2024]. Despite of its widespread applications, modeling and calculating the complex Earthatmosphere interaction are challenging. In addition, solving the differential equations is sensitive to the initial conditions, and also computationally demanding [Kochkov et al., 2024]. Deep learning-based weather prediction. Different from NWP, deep learning models learn from the historical observations to predict the weather. Though time-consuming during training, deep learning models are rapid during prediction as they do not involve solving the differential equations. State-of-the-art deep learning-based weather prediction models include FourCastNet [Kurth et al., 2023], SFNO [Bonev et al., 2023], GraphCast [Lam et al., 2023], GraphEFM [Keisler, 2022], AIFS [Lang et al., 2024], Pangu [Bi et al., 2023], Fengwu [Chen et al., 2023a], Fuxi [Chen et al., 2023b], Stormer [Nguyen et al., 2024], etc. FourCastNet and SFNO are based on the Fourier neural operator [Li et al., 2021], GraphCast, Graph-EFM and AIFS are based on the graph neural network [Wu et al., 2021], whereas Pangu, Fengwu, Fuxi, and Stormer utilize the vision [Dosovitskiy et al., 2021] and swin [Liu et al., 2021] transformers. Among these models, SFNO, GraphCast, Graph-EFM, and AIFS take the Earth's topology into consideration. Nevertheless, all of them disregard the underlying physics information.

Deep learning-based, physics-assisted weather prediction. Integrating the differential equations with deep learning models significantly improves the precisions, efficiency and robustness of the latter. Notable recent works along this line include ClimODE [Verma et al., 2024] and NowcastNet [Zhang et al., 2023]. Different to PASSAT, ClimODE characterizes the evolution of the weather variables with the continuity equation, other than the advection equation. On the other hand, ClimODE updates the velocity fields with a neural network, other than the Navier-Stokes equation. NowcastNet focuses on regional precipitation nowcasting, while PASSAT focuses on global, multi-variable and medium-term weather prediction. Besides, PASSAT solves the differential equations and trains its graph neural network on a spherical manifold, other than on the planar latitude-longitude grid used by ClimODE and NowcastNet, effectively avoiding the distortions.

3 Methods

Considering the attributions of the weather evolution demonstrated in Figure 1, we accordingly build a physics-assisted and topology-informed deep learning model for weather prediction, abbreviated as PASSAT. Given any initial time, PASSAT: (i) generates the initial velocity fields of the weather variables with the velocity branch of a spherical graph neural network; and then autoregressively (ii) predicts the effects of the Earth-atmosphere interaction with the interaction branch of the spherical graph neural network; (iii) numerically solves the advection equation on the spherical manifold; (iv) numerically updates the velocity fields through solving the NavierStokes equation on the spherical manifold, aided by the initial velocity fields provided by (i). In the following, we discuss how PASSAT captures the evolution of the weather variables and their velocity fields, via integrating the two differential equations and the spherical graph neural network (see also Figure 3).

We disregard the impact of vertical actions and focus on analyzing the advection and Navier-Stokes equations on the spherical manifold. All the analyses and approaches presented below can be readily extended to scenarios where the vertical actions are taken into account.

We begin by introducing the spherical manifold in Section

3.1 and describing the evolution of the weather variables in Section 3.2. Then, we present the advection equation on the spherical manifold, the spherical graph neural network and the Navier-Stokes equation on the spherical manifold in Section 3.3, Section 3.4 and Section 3.5, respectively. We also introduce the time integration scheme in Section 3.6. Finally, we summarize in Section 3.7.

3.1 Spherical Manifold

The historical observations used during training most deep learning-based weather prediction models are often on planar latitude-longitude grids, other than on the spherical surface of the Earth. Ignoring this topology information leads to remarkable distortions in both the neural networks and the differential equations [Cohen et al., 2018; Mai et al., 2023]. In order to avoid such distortions, we project the weather variables from a planar latitude-longitude grid onto the Earth's surface. We assume the Earth's surface to be an ideal unit sphere, with the radius of 1 unit length (6371km).

We denote the unit sphere S = {s ∈ R3 | ||s||2 = 1} as the Earth's surface. Any spatial coordinate s on the unit sphere corresponds to a point (ϕ, θ) within the planar latitude-longitude grid, where θ is the latitude and ϕ is the longitude. Thus, we use s and s(ϕ, θ) interchangeably. Given any spatial coordinate s, eϕ(s) ∈ R3 and eθ(s) ∈ R3 are two orthogonal unit vectors originated from s and along the parallel and meridian directions, respectively. We denote ∇s as the spatial gradient on the unit sphere and · as the inner product.

3.2 Evolution of Weather Variables

Weather prediction depends on understanding the evolution of weather variables that we are interested in. Given any weather variable u, its evolution is characterized as follows.

The weather variable u is viewed as a differentiable, realvalued function u : T × S → R, within which T is the time set and S is the Earth's surface. According to Figure 1, the evolution of weather variable u is attributed to the advection process and the Earth-atmosphere interaction. To be specific, for any (t, s) ∈ T × S, we have:

where δt > 0 is a small lead time, δs = δt × v(t, s), v is the velocity fields of u, and h is the tendency of u due to the Earth-atmosphere interaction. Taking the first-order Taylor's approximation of (1) yields:

According to (2), the total tendency of weather variable u can be decomposed into two part: (i) -v · ∇su, the tendency due to the advection process and (ii) h, the tendency due to the Earth-atmosphere interaction. Once the total tendency ∂u ∂t (t, s) is known, we can predict the value of u at any future time t +∆t via using proper numerical methods to solve:

In PASSAT, we use Euler's method for this time integration.

Therefore, the key of weather prediction is to compute the tendencies of the advection process and the Earth-atmosphere interaction. Though the tendency of the advection process can be numerically estimated by solving the advection equation on the spherical manifold, the tendency of the Earthatmosphere interaction is difficult to model and calculate so that we resort to a spherical graph neural network. We introduces them one by one in the following.

3.3 Advection Equation on Spherical Manifold

The advection process is the evolution of the weather variables driven by their velocity fields. Given any weather variable u, its velocity fields v : T × S → R3 are differentiable functions of time and spatial coordinate. As we disregard vertical actions, the velocity fields can be express by v(t, s) = vθ(t, s)eθ(s) + vϕ(t, s)eϕ(s), where vθ and vϕ are the velocities of u along the meridian and parallel directions, respectively. With particular note, at any initial time t and spatial coordinate s, u(t, s) is known but v(t, s) is to be calculated.

As discussed in Section 3.2, the tendency of u due to the advection process is given by solving the advection equation [Chandrasekar, 2022], as:

Once the advective derivative is known, the tendency of u due to the advection process is known too. On the spherical manifold and the planar latitude-longitude grid, the advective derivative has different forms, and the latter brings distortions in weather prediction, as discussed in the following.

Given a spatial coordinate s = s(ϕ, θ) ∈ S, on the spherical manifold, the advective derivative is in the form of [Lions et al., 1992]:

For vθ(t, s) and vϕ(t, s), PASSAT will estimate their initial values utilizing the velocity branch of a spherical graph neural network, and calculate their future values through solving the Navier-Stokes equation. The differentials ∂u ∂θ (t, s) and ∂u ∂ϕ (t, s) can be estimated using the difference quotients of u on the planar latitude-longitude grid.

In contrast, on the planar latitude-longitude grid, the advective derivative is in the form of:

where v′θ (t, s) and v′ϕ (t, s) are respectively the velocities along the meridian and parallel directions, but on the latitudelongitude planar grid, not on the spherical manifold. We have v′θ (t, s) = vθ(t, s) and v′ϕ (t, s) = vϕ(t,s) cos θ .

ClimODE and NowcastNet both calculate the the advective derivative according to (6), through estimating v′θ (t, s) and v′ϕ (t, s) with neural networks. However, we can observe that fixing the value of vϕ(t, s), v′ϕ (t, s) is not spatial-invariant - it is large when s is close to the poles and small when s is close to the equator. Such distortions will affect the pattern recognition of the neural networks. In contrast, PASSAT takes advantages of the spherical manifold, and thus avoids the distortions.

3.4 Spherical Graph Neural Network

As discussed above, to calculate ( ∂u ∂t (t, s))advection, the tendency of u due to the advection process, we need to estimate the initial velocity fields of v(t, s). On the other hand, we also need to estimate ( ∂u ∂t (t, s))interaction, the tendency of u due to the Earth-atmosphere interaction. We train a spherical graph neural network to estimate these values.

The spherical graph neural network consists of a backbone model along with two branches: the interaction branch that estimates ( ∂u ∂t (t, s))interaction and the velocity branch that estimates the initial velocity fields of v(t, s); see Figure 4. Their basic block has two components: node-edge connection and node-node aggregation. The node-edge connection component enables efficient message passing between nodes and edges [Zhou et al., 2020], while the node-node aggregation component performs graph convolution among nodes [Kipf and Welling, 2016].

The spherical graph neural network incorporates the topology information from the spherical manifold, and thus avoids the distortions caused by the planar latitude-longitude grid. For more details, please refer to the supplementary material.

3.5 Navier-Stokes Equation on Spherical Manifold

To calculate the future ( ∂u ∂t (t, s))advection, we still need to estimate the future velocity fields of v(t, s), for which we resort to the Navier-Stokes equation [Lions et al., 1992]. On the spherical manifold, the velocity fields of each weather variable v(t, s) = vθ(t, s)eθ(s) + vϕ(t, s)eϕ(s) satisfy:

We omit the pair (t, s) for notational simplicity. In the NavierStokes equation, ρ(t, s) is the atmospheric density, ω = 0.2618 (radian/hour) is the Earth's rotation speed, p(t, s) is the atmospheric pressure, and µ is a constant related to the Reynolds constant. For computational efficiency, we simplify the Navier-Stokes equation by retaining only the viscous friction in the Laplacian.

The Navier-Stokes equation governs the evolution of both vθ(t, s) and vϕ(t, s). After calculating ∂vθ(t,s) ∂t and ∂vϕ(t,s) ∂t from the Navier-Stokes equation, we apply numerical methods to predict vθ(t +∆t, s) and vϕ(t +∆t, s) as in (3).

3.6 Time Integration Scheme

Up to now, at time t, we have known ( ∂u ∂t (t, s))interaction (from the spherical graph neural network) and ( ∂u ∂t (t, s))advection (from the advection equation) with the aid of v(t, s) (from both the spherical graph neural network and the NavierStokes equation). Therefore, we can the predict the value of u at a future time t+∆t according to (3). However, the numerical methods to solve (3) are sensitive to the integration step size. As we will see, in our weather prediction, ∆t ranges from 6 to 144 hours, at the temporal resolution of 6 hours. Hence, choosing a proper integration step size is critical to medium-term or long-term prediction.

With the above consideration, in PASSAT, we set the integration step size as 0.2 hours. However, such a small integration step size requires the interaction branch of PASSAT to frequently estimate ( ∂u ∂t (t, s))interaction - see (2) and (3) - resulting in excessive back propagation during the training. To address the issue of memory-intensive training, we estimate ( ∂u ∂t (t, s))interaction only once every hour and keep it unchanged within the next hour.

3.7 Summary of PASSAT

We summarize PASSAT in Algorithm 1. For simplicity, we omit the spatial coordinate s, using ut to denote any weather variable at time t and vt to denote its velocity fields. We also use ut to denote all weather variables at time t. We use fvel and fint to denote the velocity and interaction branches of the spherical graph neural network, respectively. We use it to denote the estimated effect of the Earth-atmosphere interaction for time t and spatial coordinate s. Within the Navier-Stokes equation, 1ρ ∂pt ∂ϕ and 1ρ ∂pt ∂θ are unknown. We replace themwith the gradients of geopotential zt at the 500hPa pressure level, converting the units fromm2 second-2 to (6731km)2 hour-2 . To ensure stability, the initial velocity fields estimated from the velocity branch, with the unit of (6731km)/hour, are projected onto [-0.005, 0.005].

4 Experiments

Data & Tasks. We conduct the experiments on the European Centre for Medium-Range Weather Forecasts Reanalysis V5 (ERA5) 5.625◦ -resolution data set, spanning from 1979 to 2018 and provided by WeatherBench [Hersbach et al., 2020; Rasp et al., 2020]. The data samples from 1979 to 2015 are used in the training set, 2016 in the validation set, as well as 2017 and 2018 in the test set. The interested weather variables are temperature at 2m height (t2m), temperature at 850hPa pressure level (t850), geopotential at 500hPa pressure level (z500), u component of wind at 10m height (u10), and v component of wind at 10m height (v10).

We use PASSAT and the baseline models to predict these weather variables, at a temporal resolution of 6 hours (6am, 12am, 6pm, and 12pm of each day) and lasting for 24 steps (144 hours). Performance metrics include root mean square error (RMSE) and anomaly correlation coefficient (ACC). We release an open-source Pytorch implementation of PASSAT online 1 .

Baseline deep learning models. We compare PASSAT with the following baseline deep learning models: (i) ClimODE [Verma et al., 2024]; (ii) FourCastNet [Kurth et al., 2023]; (iii) Pangu [Bi et al., 2023]; (iv) GraphCast [Lam et al., 2023]; (v) SFNO [Bonev et al., 2023]. For fair comparisons, we unify the number of parameters of all models to the same magnitude (around 1.15 million) and train these baseline deep learning models from scratch according to their open-source codes and NVIDIA's Modulus 2 .

We do not compare with NowcastNet, Graph-EFM, AIFS, Stormer, Fengwu, and Fuxi. Among them, NowcastNet integrates the differential equations with the deep learning model, Graph-EFM and AIFS both take the Earth's topology into consideration, while Stormer, Fengwu and Fuxi do not utilize physics and topology information. However, NowcastNet exclusively focuses on regional precipitation nowcasting, while PASSAT focuses on global, multi-variable and medium-term weather prediction. Graph-EFM and AIFS adopt hierarchical grid-mesh graph structures, similar to GraphCast. Stormer, Fengwu and Fuxi utilize attention-based structures similar to Pangu, and their focus is on improving long-term predictions via enhancing the training and inference strategies.

Baseline NWP models. PASSAT is also compared with the following operational NWP models: IFS T42 and IFS T63 [Rasp et al., 2020]. IFS T42 and IFS T63 are the Integrated Forecast System (IFS) model run at two different resolutions, 2.8◦ and 1.9◦ respectively. We can observe that they are both finer than the 5.625◦ resolution of PASSAT, at the cost of being computationally demanding in solving large systems of differential equations.

Results. As demonstrated in Figure 5, PASSAT outperforms the other deep learning models in all weather variables across different lead times. The closest with PASSAT is GraphCast, which takes the topology of the Earth's surface into consideration. However, GraphCast ignores the physics of the weather evolution, and thus has to use a more complex graph structure than PASSAT (twice in terms of the number of nodes and three times in terms of the number of edges). ClimODE, despite of its physics-assisted structure, does not perform well. This phenomenon could be attributed to the following reasons: (i) ClimODE characterizes the evolution of the weather variables with the continuity equation, without considering the Earth-atmosphere interaction; (ii) ClimODE updates the velocity fields with a neural network, other than the NavierStokes equation; (iii) ClimODE ignores the topology information, and thus suffers from the distortions. In contrast, The RMSEs and ACCs of IFS T42 and IFS T63 are from [Rasp et al., 2020], only including t2m, t850 and z500 for the lead times of 72 and 120 hours. Observe that PASSAT outperforms IFS T42, a pure physical model solved at a finer resolution (2.8◦ ). Improving the resolution of the physical model from 2.8◦ to 1.9◦ , IFS T63 surpasses PASSAT and the other deep learning models, nevertheless at the cost of high computational complexity.

1 https://github.com/Yumenomae/PASSAT 5p625

2 https://github.com/NVIDIA/modulus/tree/main/modulus/models

Figure 6: Comparison between PASSAT and the three variants. The x-axis represents the lead time in hours. Smaller RMSE and larger ACC values indicate better performance.

5 Ablation Studies

We conduct ablation studies to evaluate the effectiveness of the physics and topology information used in PASSAT. We compare with three models: (i) PASSAT (without topology), which constructs the graph neural network on the latitudelongitude planar grid, other than on the spherical manifold; (ii) PASSAT (without physics), which uses the spherical graph neural network to predict the weather in an end-to-end manner; (iii) PASSAT (without topology and physics), which uses the planar graph neural network to predict the weather in an end-to-end manner.

The results are depicted in Figure 6. PASSAT significantly outperforms the three variants across all weather variables, highlighting the importance of incorporating both physics and topology information. An interesting observation from our ablation studies is that different variables benefit from different sources of information. For t2m, the performance gains of respectively using the topology information and the physics information are almost the same. On the other hand, the rest variables of t850, z500, u10, and v10 benefit more from the topology information than from the physics information.

We do not evaluate the individual effects of the advection equation and the Naiver-Stokes equation, the two parts of the physics information. First, without the advection equation, it is unnecessary to update the velocity fields with the Naiver-Stokes equation. Second, estimating the future velocity fields with the spherical graph neural network, other than the Naiver-Stokes equation, shall require frequent calls of the velocity branch, leading to excessive back propagation and thus unaffordable memory consumption during training.

6 Conclusions and FutureWorks

In this paper, we propose PASSAT, a novel physics-assisted and topology-informed deep learning model for weather prediction. PASSAT seamlessly integrates the advection equation and the Navier-Stokes equation that govern the evolution of the weather variables and their velocity fields, with a graph neural network that estimates the complex Earth-atmosphere interaction and the initial velocity fields. PASSAT also takes the topology of the Earth's surface into consideration, during solving the equations and training the graph neural network. In the 5.625◦ -resolution ERA5 data set, PASSAT outperforms both the state-of-the-art deep learning-based weather prediction models and the operational numerical weather prediction model IFS T42. Our ablation studies demonstrate that both the physics information and the topology information are essential to the performance gain.

As future works, we will extend PASSAT in the following aspects: (i) Enhance PASSAT by incorporating more weather variables; (ii) Refine PASSAT through training over a data set with a finer resolution; (iii) Incorporate new time integration scheme that is more efficient than Euler's method, during both training and prediction. We expect that PASSAT is able to motivate more research efforts in combining physics, topology and historical observations for weather prediction.

Ethical Statement

There are no ethical issues.

Acknowledgments

Corresponding author: Qing Ling. This work is supported by Guangdong Basic and Applied Basic Research Foundation (2024B1515520001).