Unveiling the Power of Noise Priors: Enhancing Diffusion Models for Mobile Traffic Prediction

1 Introduction

Communication networks serve as the essential infrastructure of smart cities, facilitating data exchange and supporting various applications. With the rapid growth of network traffic driven by advanced communication technologies and the proliferation of smart devices, efficiently managing mobile traffic resources has become more challenging [Sarker et al., 2021]. Accurate traffic prediction is crucial for optimizing resource allocation, allowing operators to adjust bandwidth and base station power in response to fluctuating demand, thereby preventing congestion and reducing energy consumption [Yu et al., 2022].

∗Corresponding Author.

The full version is at https://arxiv.org/abs/2501.13794.

Mobile traffic prediction involves forecasting the dynamics of time series and spatio-temporal data. Traditional deep learning methods-such as CNNs [Li et al., 2018; Zhang et al., 2017], RNNs [Wang et al., 2017; Lin et al., 2020], GCNs [Zhao et al., 2019; Bai et al., 2020], and transformers [Yuan et al., 2024a; Yuan et al., 2024b] -have achieved notable success in this area, often employing a deterministic approach that directly maps inputs to outputs. Recently, diffusion models [Ho et al., 2020; Nichol and Dhariwal, 2021; Yuan et al., 2024c] have emerged as a promising alternative. Unlike traditional models [Yi et al., 2024; Zhou et al., 2024], they offer a probabilistic framework that captures the intricate distributions inherent in temporal data, effectively handling uncertainties and complexities. For time series and spatio-temporal data, most diffusion modelbased solutions [Tashiro et al., 2021; Shen and Kwok, 2023; Yuan et al., 2023] naively adopt the noise modeling in the image domain and design specialized denoising networks. This noise is typically modeled as independent and identically distributed (i.i.d.) Gaussian noise, which guides the denoising process step-by-step.

However, mobile traffic data exhibits unique dynamics that differ significantly from those in images. Originating from human movements and activities, it naturally reflects human rhythms, with temporal dynamics often showing periodicity and abrupt changes [Xu et al., 2016a; Xu et al., 2016b]. Existing diffusion model-based solutions mainly focus on designing innovative denoising networks or conditioning mechanisms to incorporate various features [Tashiro et al., 2021; Shen and Kwok, 2023]. While conditioning features for the denoising network can provide some context for predictions, the noise learned in the denoising process step by step plays a more critical role [Ge et al., 2023; Zhang et al., 2024]-it directly shapes the model's generative process, significantly influencing the quality and accuracy of predictions. Despite its importance, the impact of noise in the denoising process for temporal data remains largely unexplored.

Therefore, we are motivated to address an essential research question: can the noise estimation process be redesigned to effectively incorporate the inherent characteristics of flow data? However, it is not a trivial task. First, the assumption that the noise to be predicted is Gaussian provides essential properties that ensure the stability and convergence of diffusion models. Manipulating this noise requires preserving these characteristics to maintain the model's reliability. Second, mobile traffic data is highly non-stationary, exhibiting fluctuations driven by diverse factors like human activity patterns and environmental changes. As a result, the model must capture regular, predictable patterns while remaining sensitive to sudden changes.

To address these challenges, we introduce NPDiff, a novel Noise Prior framework for Diffusion models in mobile traffic prediction, which leverages the patterns of data dynamics as priors to estimate noise, as illustrated in Figure 1. The key idea is to decompose the noise into two components: (1) noise prior captures the intrinsic dynamics of mobile traffic data, including periodic patterns and local variations, which provides a basic reference for the noise estimation in each step; (2) noise residual accounts for the additional variations that the noise prior cannot fully represent, enabling the model to adapt to more complex and unpredictable aspects of the data. Based on theoretical derivations, we can successfully reconstruct mobile traffic data through the backward diffusion process using the noise prior and the noisy data. As a general solution, NPDiff can be seamlessly integrated into existing diffusion-based models, boosting their prediction performance and adaptability to the unique characteristics of mobile traffic. Our contributions are summarized as follows:

• To the best of our knowledge, we are the first to highlight noise's role in diffusion models for mobile traffic prediction, offering new insights to apply diffusion models in this domain.
• We propose NPDiff, a general framework for diffusionbased mobile traffic prediction. By leveraging the intrinsic dynamics of crowd data as noise priors, it can be seamlessly integrated into advanced diffusion models.
• Extensive experiments across various prediction tasks and different denoising networks demonstrate that NPDiff boosts prediction accuracy by over 30%. Moreover, NPDiff improves training efficiency and robustness while reducing prediction uncertainty, positioning it as an efficient and versatile enhancement to current methodologies.

2 RelatedWork

2.1 DiffusionModels for Spatiotemporal Data

Diffusion models [Nichol and Dhariwal, 2021; Ho et al., 2020] have become popular for time series [Tashiro et al., 2021; Shen and Kwok, 2023] and spatio-temporal data [Wen et al., 2023; Yuan et al., 2024d] due to their strong generative capabilities. These models utilize additional data, such as historical observations [Shen and Kwok, 2023] or knowledge graphs [Zhou et al., 2023], to guide the denoising process. Currently, most diffusion-based approaches [Tashiro et al., 2021; Shen and Kwok, 2023; Wen et al., 2023; Yuan et al., 2024d; Yuan et al., 2023] focus on the conditioning mechanism, with limited attention to optimizing the noise component. We address this gap by investigating the impact of noise and designing a noise prior framework, leading to more accurate predictions.

2.2 Noise Manipulation in VideoModels

Diffusion models [Ho et al., 2022; Blattmann et al., 2023] have made significant progress in video generation, but generating videos with consistency remains a core challenge [Xing et al., 2023; Chang et al., 2024]. Recently, many studies [Chang et al., 2024; Qiu et al., 2023; Ge et al., 2023; Zhang et al., 2024] have enhanced the video coherence by manipulating noise during the diffusion process. Similar noise is sampled for each video frame in the early stage of diffusion [Qiu et al., 2023; Ge et al., 2023]. New forms of noise representation are also introduced [Chang et al., 2024]. Moreover, some researchers use historical images as references to provide static noise prior for the diffusion process [Zhang et al., 2024]. Compared with video data that requires temporal consistency, mobile data are much more dynamic and complex. How to design effective noise priors for such data remains unexplored yet is a promising direction.

3 Preliminaries

3.1 ProblemFormulation

Mobile traffic prediction uses historical data to predict future traffic flows. The format of mobile traffic data is characterized as a three-dimensional tensor X ∈ RT×K×C , where T is the temporal length, K is the number of spatial locations, and C denotes traffic flow features. Specifically, our task is to learn a predictive model F, which, given historical context data Xco = Xt-H+1:t of length H, predicts the traffic flow Xta = Xt+1:t+M for the next M steps, which can be represented as Xta = F(Xco ).

3.2 Diffusion-basedMobile Traffic Prediction

For mobile traffic prediction, the diffusion model predicts target values conditioned on historical data xco 0 . During the forward process, noise is progressively added to the target data with the noise schedule {βn}Nt=1 . Given αn = 1 - βn and ¯αn = ∏ni=1 αi, the noised target at any diffusion step can be calculated using the formula below:

Reversely, the denoising process is formulated as follows:

(a) Visualization of Noise Maps (b) Comparison of the Same Time

Points across Two Weeks

Figure 2: Noise visualization of the MobileNJ dataset. (a) Noise in 12 consecutive time points during the denoising process, encompassing five distinct diffusion steps from step 10 to step 45. (b) Noise comparison of the same timestamps across two consecutive weeks.

The model is trained to estimate the noise ϵ. The model's parameters θ are optimized by the following loss function:

4 Observations and Analysis

To better demonstrate the role of noise in the denoising process, we visualize the estimated noise for different diffusion steps and timestamps using MobileNJ dataset. Specifically, we obtain the noise maps through the Denoising Diffusion Implicit Models (DDIM) inversion process and employ tdistributed Stochastic Neighbor Embedding (t-SNE) to visualize them. The results are shown in Figure 2, the diamond squares denote the noise estimated by the model during this process while the other small dots in the background represent Gaussian noise, serving as a reference noise.

Figure 2(a) illustrates that, for a specific time point, the noise across different diffusion steps exhibits high continuity, approximately following a linear trend. This suggests that noise at every diffusion step is constrained by certain inherent conditions. Additionally, the noise shows a counterclockwise rotation trend, aligning with the temporal dimension. In Figure 2(b), we further observe that the noise from the same time point one week later highly overlaps with those from the current time point, revealing clear periodic characteristics. This periodicity is highly consistent with the inherent dynamics of mobile traffic data, indicating a strong connection between temporal dynamics and noise within diffusion process.

These findings underscore the critical role of noise in the denoising process, as the noise exhibits clear and consistent patterns. Consequently, integrating the intrinsic dynamics of temporal data into the modeling of noise is well-motivated and promising for enhancing the model's effectiveness.

5 Method

5.1 Framework Overview

We introduce the NPDiff framework, which incorporates the mobile traffic dynamics as noise prior to the diffusion process. Figure 3 illustrates an overview of our approach. Figure 3(a) shows the extraction of two key data dynamics: periodic dynamics and local dynamics, while Figure 3(b) illustrates the denoising process with noise priors. The noise is estimated through two pathways: (1) directly calculating the noise prior based on data dynamics; and (2) predicting the residual noise using the denoising network.

5.2 Dynamics of Mobile Traffic

By analyzing the dynamics of mobile traffic data, we select two representative dynamics to calculate noise priors, which we define as periodic dynamics and local dynamics, respectively. Periodic dynamics capture the regular patterns, while local dynamics reflect short-term fluctuations. We calculated the cosine similarities between these dynamics and actual values. As shown in Table 1, they both achieve high cosine similarities of around 0.8 across all datasets.

Periodic Dynamics. To capture periodic dynamics, we leverage Fast Fourier Transform (FFT), which is widely used in time series analysis. Specifically, we select several key components obtained by using FFT and convert them back to the time domain. This process can be formulated as follows:

where A(k) is the amplitude of the k-th frequency component obtained by applying FFT to the training data x, and Φ(k) represents the phase of the k-th frequency component. L represents the length of the training set. κ(1) , · · · , κ(K) are the the top K amplitudes selected by arg-TopK. NK represents the number of selected components, which is set to 5 or determined by the number of components with amplitudes exceeding the average amplitude in our paper. fκ(k) refers to the frequency corresponding to the k-th component, and ¯ fκ(k) , ¯Φκ( k) represent the corresponding conjugates. We then compute the average values of the time-domain signal at corresponding time points across different cycles within a specified period length P to extract the final dynamics. It can be formulated as follows:

where NP denotes the total number of complete periods in the training data, t+kP represents the sampled value at time t within each period P . We extend periodic dynamics Dp[t] within period P to the entire dataset to obtain the dynamics corresponding to each time point. The period P is defined as one week in most datasets, while P is set to one day for MobileBJ dataset due to the limited data duration. If the dataset is sufficiently large, P can be chosen to represent longer periods, such as a month or a year, which correspond to the behavioral patterns of human activity.

Local Dynamics. Periodic dynamics reflect the periodic patterns, but they may not be well-suited for capturing short-term variations. Mobile traffic data usually depend on consecutive time steps, the current values are strongly correlated with the nearby last step. Therefore, we incorporate local dynamics. Specifically, we utilize the lag-1 values to depict local variations. Mathematically, this can be described as:

5.3 Derivation and Fusion of Noise Priors

As analyzed in Section 4, there is a strong correlation between noise and data dynamics. Therefore, our objective is to enhance the model's performance by leveraging these correlations. In this section, we describe the method for calculating noise priors and explain how they are integrated into the denoising process. For an intuitive representation of the diffusion process and the use of noise priors, please refer to Appendix Figure 9.

Derivations of Noise Priors

Given the extracted periodic and local dynamics, a basic assumption is that they are closely aligned with the corresponding target values. Based on this assumption [Zhang et al., 2024], we can reformulate the mobile traffic data as follows:

where ∆x represents a small error term that quantifies the difference between the target value and the extracted dynamics. We then rearrange Eq. (1) to derive the formula below:

This noise ϵ serves as the target noise we need to estimate during the training process. Next, we substitute Eq. (7) into Eq. (8), and obtain:

Thus, the noise prior can be defined as follows:

According to Eq (10), the noise prior can be directly computed at any diffusion step based on data dynamics. And the target noise can be written as:

where ∆ϵ represents a small term, we term it 'residual noise'.

Fusion of Noise Priors

According to Eq. (7) and Eq. (9), when the dynamics are close to the target values (i.e., ∆x is small), the noise prior ˜ϵ can serve as a reference noise to ϵ. Considering that the extracted prior dynamics represent regular patterns and may not always be accurate enough when facing irregular changes, we still need the denoising network to estimate the residual noise. So we define the final noise ̂ϵ as a weighted combination of the noise prior ˜ϵ and the model-predicted noise ϵθ(xta n , n|xco 0 ).

This combination is controlled by an adjustable hyperparameter λ that balances the contributions of the noise prior and

By applying Eq. (12), the noise prior is combined with the model's output, allowing the integration of data dynamics by manipulating the noise in the diffusion process.

5.4 Training and Inference

We provide the detailed algorithmic procedures for training and inference in Appendix A.4. Different from the vanilla diffusion models, the optimization objective in the training phase has shifted from Eq. (3) to:

In the sampling phase, the expression of µθ(xta n , n|xco 0 ) in Eq (2) is reformulated as:

5.5 Flexibility of NPDiff

NPDiff offers flexibility and universality by seamlessly integrating with state-of-the-art denoising network architectures. To showcase its adaptability, we employ three representative models from the spatio-temporal domain as the denoising networks: CSDI [Tashiro et al., 2021], ConvLSTM [Shi et al., 2015], and STID [Shao et al., 2022]. These models were carefully selected to represent a diverse spectrum of deep learning paradigms. Specifically, CSDI leverages transformers as its denoising backbone, ConvLSTM incorporates Convolutional LSTM structures, and STID employs a multi-layer perceptron (MLP)-based architecture. Together, these models encompass the core building blocks of modern deep learning-namely CNNs, LSTMs, MLPs, and transformers-ensuring that NPDiff's applicability extends across various network modules.

6 Evaluations

6.1 Experimental Settings

We provide a detailed description of the datasets, baselines, and experimental configurations in Appendix A.

Datasets. We evaluate our method on four large-scale cellular network traffic datasets: MobileBJ, MobileNJ, MobileSH14 and MobileSH16, which are sourced from three major cities in China: Beijing, Nanjing, and Shanghai.

Baselines. We select 13 state-of-the-art models as our baselines, which are divided into three categories: Classic models include HA and ARIMA, which can be applied to time series prediction without extensive training. Time series prediction models include PatchTST [Nie et al., 2022], iTransformer [Liu et al., 2023], and Time-LLM [Jin et al., 2023], which are state-of-the-art models for multivariate time series forecasting. Deep spatio-temporal prediction models consist of two groups: (i) urban spatio-temporal models, including STResNet [Zhang et al., 2017], ATFM [Liu et al., 2018], STNorm [Deng et al., 2021], STGSP [Zhao et al., 2022], TAU [Tan et al., 2023], and PromptST [Zhang et al., 2023]; (ii) video prediction models, including MAU [Chang et al., 2021] and MIM [Wang et al., 2019], considering that the format of mobile traffic data is similar to video data.

6.2 Overall Performance

Our experiments include both multi-step and one-step prediction tasks. The multi-step prediction evaluates capabilities of long-term prediction, while the one-step prediction task assesses the model's ability to tackle abrupt changes.

Multi-step Prediction. For the main experiment, we use 12 historical steps to predict the next 12 steps, which is one of the most typical setups in mobile traffic prediction. It is worth noting that different datasets have different temporal resolutions, so the time length corresponding to 12 steps differs. Table 2 shows the prediction results of NPDiff with CSDI (results for ConvLSTM and STID are shown in Appendix Table 5). Specifically, we have the following observations:

• Video prediction model performs poorly on mobile traffic data. MAU and MIM, show relatively poor performance compared to other models, indicating that the dynamics of video data differ from those of mobile traffic data. Although video data and mobile traffic data share structural similarities, directly utilizing video prediction models delivers unsatisfying performance.
• Our proposed noise prior framework achieves superior performance. Our proposed approach achieves the best performance compared to competitive baselines across almost all datasets. Compared with CSDI without noise prior, it has significant improvements in most cases, with an average reduction of 41.6% in MAE and 54.4% in RMSE. The remarkable performance improvement highlights the great potential of manipulating noise in diffusion models.
• More accurate prior dynamics boost more performance gain. Specifically, on the MobileBJ and MobileNJ datasets, the diffusion models' average performance improves by 40.6% and 47.2% in terms of MAE when using the noise prior. Meanwhile, on the MobileSH14 and MobileSH16 datasets, although the improvements are slightly lower, there is still an average performance increase of 21.1% and 25.5% in MAE. The observations align with the cosine similarity results shown in Table 1, where the similarity scores for MobileBJ and MobileNJ datasets are higher.

One-step Prediction. For this task, we set the context length to 12 and limit the prediction horizon to 1. Specifically, we use periodic dynamics and local dynamics, separately, to investigate these contributions to the final performance. Table 3 illustrates the results of one-step prediction. When applying periodic dynamics, significant performance gains are observed in most datasets, with an average MAE reduction of 30.3%. However, in a few datasets, it does not yield expected performance. In contrast, local dynamics proves effectiveness across all datasets and reduces the MAE by 35.1% on average. It indicates that local dynamics are more suitable for one-step prediction, due to their ability to reflect short-term abrupt and sudden changes. Also as shown in Table 1, the similarity between local dynamics and the real data is higher than that of periodic dynamics, indicating the importance of adopting suitable dynamics as noise prior.

6.3 Ablation Studies

Noise prior is the most essential design of our method. In this section, we explore the impact of two key hyperparameters associated with it: Nk and λ.

Noise Fusion Coefficient. Figure 4 shows the impact of the coefficient λ in Eq. (12). We can observe that as λ increases, the model's performance gradually improves, demonstrating the effectiveness of noise prior. This also indicates that increasing the contribution of the noise prior allows for better utilization of prior dynamics. The model achieves optimal performance when λ is around 0.5. However, as λ continues to increase, a slight performance decline occurs. This can be attributed to the fact that the extracted prior dynamics only provide regular dynamics and cannot fully represent all variations in the real data. Over-reliance on the noise prior may result in suboptimal performance.

Components of FFT. Figure 5 illustrates the impact of the number of FFT frequency components in periodic dynamics on the model's performance. The experimental results show that as the number of components increases, the performance of models improves accordingly. When the number of Fourier components reaches 5, allowing for sufficiently accurate capture of the data dynamics, the model's performance metrics tend to stabilize.

6.4 Training Efficiency and Robustness

Training Efficiency. Figure 6 illustrates the trend of performance on the validation set over the first five epochs. We can observe that the model with noise prior exhibits significantly faster convergence in the early stages of training. This rapid convergence can be attributed to the model's ability to directly estimate part of the noise, enabling it to reach the optimal solution more quickly during training. This efficiency improvement is important for real-time applications.

Robustness Analysis. We examine the robustness of NPDiff from two aspects:

• Different Prediction Tasks. We extend the evaluation task to a series of scenarios with different historical horizons and prediction steps, including 12-6, 24-24, 24-12, and 24-1. The detailed results can be found in Appendix Tables 6 t0 9. The results show that our method improves model performance across all prediction tasks, demonstrating its robustness in different tasks.
• Noise Perturbations. We evaluate the performance of our method under noise perturbations by introducing Gaussian noise with a mean of 0 into the mobile traffic data. The variance levels are set to range from 10% to 50% of the mean value. Figure 7 shows the performance of CSDI under different levels of noise perturbation. The results show that as the noise level gradually increases, the models incorporating noise prior exhibit higher stability, with only a minor performance decrease, proving its robustness.

6.5 Case Study

To further investigate the impact of noise prior, we visualize the prediction results for the MobileNJ dataset. We use CSDI as an example, and Figure 8 illustrates the results (the prediction results with ConvLSTM and STID can be found in Appendix Figure 10).

We observe that, compared to CSDI (indicated in gray), incorporating the noise prior (shown in blue) significantly reduces the uncertainty in predictions. This is especially important for probabilistic forecasting, where uncertainty reflects the reliability of the model's predictions. Additionally, as shown in Figure 8(a, b, and h), when dealing with uncommon circumstances, where sudden changes occur, our approach can well handle them due to the incorporation of prior dynamics.

7 Conclusion

In this paper, we propose NPDiff, a general noise prior framework for diffusion-based mobile traffic prediction, pioneering the manipulation of noise in diffusion models for temporal data prediction. NPDiff offers significant advancements over existing solutions in several key areas, including enhanced prediction accuracy, improved training efficiency, increased robustness, and reduced uncertainty. These improvements provide a more efficient and reliable approach to real-world mobile traffic management, demonstrating the framework's potential to enhance both performance and practical applicability in this critical domain.

Our work is highly extensible, offering multiple avenues for future exploration. On one hand, further research could focus on refining the fusion of the noise prior and residual noise to better handle complex data scenarios. On the other hand, designing more advanced noise priors is another promising direction. This could involve developing priors that adapt to the unique dynamics of various spatio-temporal scenarios or creating a more general prior that can generalize effectively across diverse scenarios.

Acknowledgements

This work is supported in part by the National Natural Science Foundation of China under Grants 62476152 and U24B20180.

Contribution Statement

Zhi Sheng and Daisy Yuan contributed equally to this work.