Denoising-Aware Contrastive Learning for Noisy Time Series

1 Introduction

Time series learning has attached great importance in various real-world applications [Ismail Fawaz et al., 2019], such as heart failure diagnosis and fault detection in the industry. Given the abundance of unlabeled time series data [Meng et al., 2023b], there has been a surge in attention towards time series self-supervised learning (SSL) that extracts informative representations from unlabelled time series data for downstream tasks [Ma et al., 2023]. Many time series SSL methods have been proposed in recent years [Zhang et al., 2023a], including contrastive learning-based [Tonekaboni et al., 2021], generative-based [Chowdhury et al., 2022], and adversarial-based [Luo et al., 2019] approaches.

Despite the encouraging progress made in time series SSL, the existing research often assumes that the given time series is clean, with limited discussion on the potential noise in the time series. Unfortunately, many time series (e.g., bio-signals collected from sensors) naturally suffer from noises that can severely change the data characteristics and impair representation learned by SSL algorithms [Zhang et al., 2023b]. For instance, some samples in the ECG dataset PTB-XL [Wagner et al., 2020] exhibit considerable high-frequency noise, depicted in Fig. 1(a). Directly applying the existing time series SSL methods CA-TCC [Eldele et al., 2023] and TS2vec [Yue et al., 2022] on this dataset for the classification task yields poor accuracy, as illustrated on the left in Fig.1(c). In contrast, by employing an appropriate denoising method like LOESS [Burguera, 2018], as illustrated in Fig.1(b), the accuracy experiences a significant improvement, as shown in the middle of Fig.1(c). Motivated by this, we study the following research question: How can we effectively denoise noisy time series for SSL to learn better representations?

∗Corresponding author

The de facto strategy to handle noises in time series is to apply conventional denoising methods (e.g., LOESS mentioned above) and then perform model learning [Lai et al., 2023]. However, we argue that this pre-processing approach cannot fully eliminate the effect of noise in time series. Firstly, the diverse types of noise in time series make it challenging to select the most suitable denoising methods [Zhang et al., 2021a]. Many real-world datasets, e.g., ECG data, may contain thousands of samples, where each sample involves different noises [He et al., 2015], such as high-frequency noise, baseline wandering, and muscle artifacts [Zhang et al., 2021b]. There is often no denoising method that can universally handle all types of noise [Robbins et al., 2020; Zheng et al., 2020], making it difficult to select suitable denoising methods in real-world applications. For example, on the right-hand side of Fig.1(c), the accuracy drops when we apply an unsuitable denoising method median filter. Secondly, noise can be amplified after mapping the raw data into latent space. To illustrate this, we compare the signal-to-noise ratio (SNR) [Chawla, 2011] of the raw time series, the denoised time series, and the representations learned upon the denoised time series in Fig. 1(d). The results show that (i) the SNR value of the denoised data is improved, meaning that the noise is mitigated in the raw data; (ii) however, if further mapping the denoised data into representations, the SNR value drops, which suggests that the representation learning process could amplify the noise. That is, even though the denoising methods can alleviate noise in the raw time series, the noise could "come back" in the representation space and still hamper SSL. Hence, how to mitigate the noise in time series SSL remains an open challenge.

In this paper, we propose DEnoising-aware Contrastive Learning (DECL), an end-to-end framework that can leverage any conventional denoising methods to guide noise mitigation in representations. Specifically, DECL involves two novel designs. Firstly, building upon an auto-regressive encoder, we propose a novel denoiser-driven contrastive learning objective to mitigate the noise. The key idea is to construct positive samples through the application of existing denoisers on raw time series, and concurrently generating negative samples by introducing noise into the same time series. Subsequently, through optimization using a contrastive learning objective, we guide the representations toward positive samples and distance them from negative samples, thereby reducing the noise. Secondly, we introduce an automatic denoiser selection strategy to learn to select the most suitable denoisers for each sample. Our motivation is that, in autoregressive learning, noisy data usually tend to have large reconstruction errors, and vice versa. As a result, we can use the reconstruction error as a proxy for how suitable a denoiser is to the sample. We further incorporate this denoiser selection strategy into the proposed denoiser-driven contrastive learning and optimize them jointly. We summarize our contribution below.

• Problem: Motivated by the observation that removing noise boosts the performance of SSL, we formulate the problem of self-supervised learning on noisy time series.
• Algorithm: We propose a customized denoising-aware contrastive learning method 1 that automatically selects suitable denoising methods for each sample to guide mitigating data noise in representation learning.
• Experimental Findings: Extensive experiments show the effectiveness of our method. We also verify that DECL is robust with varying degrees of noise and the learned representations have less noise.

1 https://github.com/betterzhou/DECL

2 Problem Statement

Self-Supervised Learning on Noisy Time Series. Given a set of time series D = {x1, x2, . . . , xN} of N instances with a certain amount of noise S, the goal is learning a nonlinear mapping function F in a self-supervised manner that maps each time series xi to a representation zi to best describe itself. Specifically, for a time series sample xi = [xi,1, xi,2, . . . , xi,T ] ∈ RT×d with T timestamps and d feature dimensions, it involves noise si = [si,1, si,2, . . . , si,T ] ∈ RT×d and the denoised data is present as vi = [(xi,1 - si,1), (xi,2 - si,2), . . . , (xi,T - si,T )]; the mapping function F aims to learn a representation zi = [zi,1, zi,2, . . . , zi,C] ∈ RC×r , where zi,t ∈ Rr is representation at timestamp t with r dimensions.

3 Methodology

In this section, we present the proposed DEnoising-aware Contrastive Learning (DECL), as shown in Fig. 2. It consists of three components: (i) auto-regressive learning, which is for generating informative representations in latent space; (ii) denoiser-driven contrastive learning, which exploits denoising methods to guide mitigating noise in representation learning; (iii) automatic denoiser selection, which selects suitable denoising methods for every sample in learning.

3.1 Auto-regressive Learning

The purpose is to map raw data into latent space via an encoder and exploit the obtained representations for selfsupervised learning. Specifically, it involves an encoder fenc, which is a 3-block convolutional architecture, and an autoregressive (AR) module far. For an input xi, the encoder maps it to a high-dimensional latent representation zi = fenc(xi), where zi ∈ RC×r . Then, for the representation zi, the AR module summarizes all zi,j≤t = {zi,1, zi,2, ..., zi,t} into a context vector ci = far (zi,j≤t) , ci ∈ Rh , where h is the hidden dimension of far. The context vector ci is used to predict the future timesteps from zi,t+1 until zi,t+k(1 ≤ k < C), zi,t+k ∈ Rr , where k is the number of predicted timesteps and r is the number of output channels in fenc. Here, we use Transformer [Vaswani et al., 2017] as the far, which is comprised of successive blocks of multi-headed attention followed by an MLP block. We stack L identical layers to generate the prediction. To enable using ci to predict the timesteps from zi,t+1 until zi,t+k, we adopt a linear layer parameterized by W ∈ Rh×r to map ci back into the same dimension as zi. Finally, we obtain the predicted timesteps ˆ zi,t+1 until ˆ zi,t+k. Accordingly, the reconstruction loss ei for xi can be computed by the mean square error between zi,t+j and ˆ zi,t+j :

where zi,t+j is from zi and ˆ zi,t+j is the prediction. Minimizing the reconstruction loss enables jointly learning fenc and far for generating informative representations.

3.2 Denoiser-driven Contrastive Learning

Considering that noise could be amplified in the latent space, we propose to directly eliminate noise from representations via denoiser-driven contrastive learning. Our motivation is that conventional denoising methods have been proven effective for noise removal if they are properly used [Zheng et al., 2020]; intuitively, the representations of the denoised data suffer from less noise than that of raw data and can be exploited as positive samples to guide learning. Vice versa, it is also viable to amplify noise in the raw data, and the representations of the noise-enhanced data are not desired and can act as negative samples. Following this, we propose to exploit the noise-reduced and noise-enhanced data in contrastive learning for better representations.

It involves two steps: (1) generating denoised and noiseenhanced counterparts for raw data and obtaining the representations; (2) mapping the representations of raw data close to that of the denoised ones (i.e., positive samples) and far away from the noise-enhanced counterparts (i.e., negative samples). Specifically, we first conduct data augmentation on raw data. Given a suitable denoising method ϕj for raw data xi, we can generate the denoised data x(d) i,j . As for synthesizing noise-enhanced data, the principle is to add more noise w.r.t. the corresponding noise type. To this end, we compare raw data xi with the denoised one x(d) i,j to identify what are the "noises" and then amplify them. In particular, we first obtain the data noise vi,j, which has been removed by denoising method ϕj, by using xi subtracts x(d) i,j . Then, we scale the values of vi,j to further amplify noise and later add it back to xi. In this way, we obtain the noise-enhanced counterpart x(n) i,j for raw data xi.

After data augmentation, we obtain their representations via the fenc and perform denoiser-driven contrastive learning. Given a sample xi and a denoising method ϕj , we first build a triplet of representations A(zi, ϕj) as follows:

where z(n) i,j and z(d) i,j are the representation of x(n) i,j and x(d) i,j , and are respectively taken as negative and positive sample in contrastive learning. Then, by taking representation zi as an anchor, we pull the anchor towards the positive sample while pushing it far away from the negative sample in the latent space. Furthermore, considering the triplet of {z(n) i,j , zi, z(d) i,j } consists of the representations with a descending degree of noise, the direction from large noise to trivial noise in the latent space may indicate a better denoising effect. Accordingly, we also enforce that the mapping direction of positive and negative samples to the anchor shall be opposite for each triplet. Overall, the contrastive learning loss Li,j for a data xi and a denoising method ϕj is shown as:

where ⟨·, ·⟩ denotes cosine similarity, 1za̸=zi ∈ {0, 1} is a binary indicator that equals to 0 when za denotes zi, τ is a temporal parameter, Lreg i,j is a regularization term that maps representation zi towards a noise-free direction.

3.3 Automatic Denoiser Selection

Notably, the above contrastive learning requires a suitable denoising method ϕj for sample xi. However, it is nontrivial to fulfill this requirement. Although many conventional denoising methods have been proven effective for noise removal, they may not well handle noise for a given time series xi when the noise type does not match. To address this issue, we propose to collect a set of commonly used denoising methods M = {ϕ1, ϕ2, . . . , ϕm} (shown in Appendix Table 2) in related works and automatically select suitable ones from them.

Here, we propose to exploit the reconstruction error from far to determine suitable methods in M. Our inspiration is that noisy data usually cause relatively large reconstruction errors. Following this, the data processed by suitable denoising methods would have small reconstruction errors, while the unsuitable methods would cause large errors. In this way, suitable methods in M can be automatically determined for every sample xi. However, directly using the above reconstruction error (Eq.(1)) as the learning objective may not reach the goal. This is because the AR module may overfit the noisy data, thus rendering raw data to have smaller reconstruction errors than the denoised ones. To tackle the issue, we propose another regularization term that encourages far to learn global patterns from the noisy time series and avoid overfitting. The key idea is to augment data with more noise, feed their representations to far, and enforce reconstructing the raw data that are of less noise. Specifically, we add Gaussian noise to raw data for data augmentation and map it into the latent space to get the representation z(n) i as follows:

Similarly, we feed the z(n) i,j≤t into far to obtain c(n) i and the prediction ˆ z(n) i,t+j . Formally, the overall learning objective for auto-regressive learning is:

where N is the number of training samples. During optimization, LAR encourages capturing global patterns from the noisy time series for prediction, which avoids overfitting the raw data. Thereby, the samples processed by suitable denoising methods would eventually own smaller reconstruction errors than the raw data, and it is feasible to use reconstruction errors for the automatic selection.

Considering that a data sample may contain multiple types of noises and require different denoising methods, we leverage all the suitable methods in M by assigning them large weights while setting small weights for the unsuitable ones. Specifically, we first feed the processed sample x(d) i,j by a denoising method ϕj to the optimized far and obtain the reconstruction error ei,j based on the Eq. (1). Then, we compute the weight value wi,j for ϕj with a softmax function as below:

where the m is the number of methods in M. Hence, the contrastive learning objective LCL is a weighted combination of Li, j for the denoising methods inM, i.e.,

where N is the number of the training samples, wi,j is the weight score for method ϕj . By minimizing LCL, our method encourages mapping the representations of raw data toward a noise-free direction, thus mitigating the data noise in latent space. Finally, we combine auto-regressive learning and contrastive learning for joint optimization:

,

where γ is a weight value to balance the two terms.

4 Experiments

We perform empirical evaluations to answer the following research questions: RQ1: How effective is DECL for unsupervised representation learning? RQ2: Is the method effective with fine-tuning? RQ3: What are the effects of each component? RQ4: Is it robust with varied degrees of noise? RQ5: How sensitive is it to the hyper-parameters? RQ6: How does the method work in practice?

4.1 Dataset

We employ five noisy time series datasets. SleepEDF [Goldberger et al., 2000] is an EEG dataset in which each sample records human brain activity. The data in FaultDiagnosis [Lessmeier et al., 2016] are collected from sensor readings of bearing machine under different working conditions. CPSC18 [Liu et al., 2018], PTB-XL [Wagner et al., 2020], and Georgia [Alday et al., 2020] are ECG datasets wherein each sample reflects heart activity. The data statistics are shown in Table 1. See Appendix A.1 for more details.

4.2 Experimental Settings

Comparative Methods. We compare our method with representative SSL methods, including contrastive learning-based, such as TF-C [Zhang et al., 2022], TS2vec [Yue et al., 2022], TS-CoT [Zhang et al., 2023b], and CA-TCC [Eldele et al., 2023], as well as generative-based methods, e.g., CRT [Zhang et al., 2023c] and SimMTM [Dong et al., 2023]. We provide detailed descriptions in Appendix A.2.

Evaluation Metrics. We follow previous works [Eldele et al., 2023][Yue et al., 2022] and adopt Accuracy and Weighted-F1 scores for classification performance evaluation. Implementation Details. We split the data into 40%, 20%, and 40% for training, validation, and test set. We set the learning epochs as 100 and adopt a batch size of 128 for both pre-training and downstream tasks, as we notice the training loss does not further decrease. In the transformer, we set L as 4, the number of heads as 4, and the hidden dimension size as 100. The details of the encoder and AR module can be referred to in Appendix C.2. As for the hyper-parameters, we set k as 30% of the total timestamps, assign α as 0.5, and set γ as 0.1 for all the datasets. The method is optimized with Adam optimizer; we set the learning rate as 1e-4 and the weight decay as 5e-4. We collect the commonly used denoising methods from related papers for different types of time series and apply the reported hyper-parameters or the default ones for noise removal (see the details in Appendix Table 2). For the baselines, we select the hyper-parameters with the best performance on the validation set for the downstream task. For a fair comparison, we pre-process the raw data for all the baselines by (i) applying each denoising method in M for the dataset and reporting the one with the best performance (i.e., DN), and (ii) combining all the denoising methods for noise removal (i.e., Merge). We run experiments 10 times and report the averaged results.

4.3 Linear Evaluation of Representations (RQ1)

We first pre-train SSL methods with unlabeled data for representation learning, then use a portion of labeled data (i.e., 10% and 30%) to evaluate the effectiveness of the learned representations. Following the standard linear evaluation scheme [Eldele et al., 2023], we fix the parameters of the self-supervised pre-trained model and regard it as an encoder, and then train a linear classifier (single fully connected layer) on top of the encoder. The results with 10% and 30% training labels are shown in Table 2 and Appendix Table 1. We find that existing SSL methods achieve sub-optimal performance on noisy data and the performances get boosted after applying a suitable denoising method for noise mitigation. Besides, directly combining all the denoising methods for preprocessing causes unsatisfactory performance. This can be explained by that certain denoising methods may induce extra noise into data when the noise type does not match. Additionally, DECL achieves superior performance than other SSL methods. Specifically, the performance boost over the best baseline on the CPSC18 dataset is about 3%. This is because our method exploits suitable denoising methods to mitigate noise and guide representation learning.

4.4 Fine-tuning Performance Evaluation (RQ2)

To simulate real-world scenarios where a few labeled data are accessible, we fine-tune our pre-trained model with the labels and investigate its effectiveness. There are two setups.

Fine-tuning on the Source Dataset. Following previous works [Lan et al., 2022], we pre-train the model with unlabeled data and then fine-tune it with different amounts of the training labels (i.e., 1%, 5%, 10%, 50%, 75%, and 100%) from the same dataset. Fig. 3 shows the performance comparison against its supervised learning counterpart and a strong baseline on three datasets. We observe that supervised learning performs poorly with limited labeled data (e.g., 1%), while the fine-tuned models achieve significantly better performance. It verifies that self-supervised pre-training can alleviate the label scarcity issue. Besides, DECL outperforms the strong baseline when using different ratios of training labels in fine-tuning. In brief, it shows the effectiveness of our method under the fine-tuning mode.

Fine-tuning on New Dataset. To evaluate the generalizability of the learned representations, we further pre-train the model on one dataset and perform supervised fine-tuning on another dataset. Specifically, we follow the one-to-one evaluation scheme [Zhang et al., 2022] and use a portion of labeled data (10%) for fine-tuning. Table 3 shows the results under six cross-dataset scenarios on the ECG data. Similarly, we find that combining all the denoising methods would render unsatisfactory performance for the baselines than that of applying a suitable one. Besides, our fine-tuned model consistently outperforms the strong baselines. Furthermore, pre-training on a comprehensive dataset (e.g., CPSC18) usually promises better performance on the new dataset, which is consistent with the findings from related papers [Yang et al., 2023]. Overall, the results verify that DECL can alleviate the impact of noise on learning informative representations and generalize well on cross-dataset scenarios.

4.5 Ablation Study (RQ3)

We also examine the effect of each component in DECL on its overall performance. Specifically, we derive different method variants for comparison: (1) DECL-, which drops the regularization term in the auto-regressive learning and remains others unchanged; (2) DECL*, which deletes the direction constraint in the contrastive learning; (3) CL, which assigns equal weights to the denoising methods and keeps others unchanged; (4) Only AR, which merely leverages LAR as the learning objective; (5) Only AR-, which exploits LAR without the regularization term. The results are shown in Table 2. We find that (i) merely using LAR renders unsatisfactory performance. This is because the noise in latent space cannot be eliminated and hampers representation learning. (ii) Our method outperforms CL and DECL-, illustrating that attaching equal importance to the denoising methods would hinder representation learning and the regularization term in LAR is conducive to the overall performance. (iii) Our method achieves higher performance than DECL*, showing that the direction constraint in contrastive learning helps to eliminate the noise in representation learning.

4.6 Robustness Analysis against Data Noise (RQ4)

We further investigate the robustness of DECL against varying degrees of data noise. Specifically, we induce Gaussian noise to the raw data (with zero mean and varying standard deviations) and then conduct the linear evaluation for performance comparison. The results on three datasets are shown in Fig. 4; see Appendix B.2 for more results. We have two interesting findings. (i) By inducing a larger amount of noise, the performances of the methods gradually drop. This occurs since the adopted denoising method(s) cannot eliminate the noise sufficiently, thereby impairing representation learning.

(ii) Our method consistently outperforms the strong baselines. Because DECL selects suitable denoising methods for every sample and further diminishes noise from representations with contrastive learning.

4.7 Sensitivity Analysis (RQ5)

We perform sensitivity analysis to study the main hyperparameters: the number of predicted future timesteps k in Eq. (5), the weight α in Eq. (3), and the weight γ in Eq. (8). Specifically, we adopt the same setup as the linear evaluation experiment, present the results of the PTB-XL dataset in Fig. 5, and show more results in Appendix B.3. We first analyze the impact of k, where the x-axis denotes the percentage k/C and C is the total number of timesteps. It shows that, as the percentage value increases, the performance first boosts and then declines. Hence, we suggest setting it to the scope of 0.1-0.4 in practice. Regarding the hyper-parameter α and γ, when raising its value, the performance first rises and later declines. It occurs because either a small or a large value fails to achieve a balance between the learning objectives. Given this, we recommend setting the value of α to 0.1-5 and the value of γ to 0.1-1.

4.8 Visualization Results (RQ6)

The Effect of the Regularization Term. Here, we examine whether the proposed regularization term in LAR (see Eq. (5)) can alleviate the overfitting issue and enable the reconstruction error as an indicator for choosing suitable denoising methods. Specifically, we compare the model learned with LAR (i.e., with regularization) and the counterpart without regularization by visualizing the distribution of reconstruction errors on the PTB-XL dataset. See Appendix B.4 for more details. The results in Fig. 6 show that (i) without a regularization term, the distributions of reconstruction errors between raw and denoised data become similar, rendering it hard to use the reconstruction error as the indicator. (ii) After adding the regularization term, the distributions of reconstruction errors are more distinguishable. In brief, it verifies that the proposed regularization term indeed alleviates overfitting and benefits determining suitable denoising methods.

Analysis on the Selected Denoising Methods. We also verify whether DECL can assign more weights to the suitable denoising methods. In detail, we adopt a case study on the PTBXL dataset to (i) visualize the weight values of the denoising methods in M for a sample with high-frequency noise, and (ii) showcase a raw data and the denoised counterparts to examine the denoising effect. As shown in Fig. 7(a), the AR module assigns different weights on the set of potentially feasible denoising methods. For example, the LOESS method, which is reported to remove high-frequency noise for ECG [Burguera, 2018], obtains large a weight value; whereas the PCA method, which does not suit this type of noise [Alickovic and Subasi, 2015], owns a small value. We further visualize the denoised data by LOESS in Fig. 7(b), which demonstrates that noises are well mitigated. However, as shown in Fig. 7(c), the denoised data by PCA still involve severe noises. More results are presented in Appendix B.5. The above analyses prove that our method can automatically select suitable denoising methods for the noisy time series.

Analysis on the Learned Representations. Further, we analyze the effect of DECL on representations. Similar to the pilot study, we induce some Gaussian noise to the raw data and compare the SNR value of the randomly initialized representations and that of the learned representations after optimization. The result on the PTB-XL dataset is presented in Fig. 7(d). We observe that the SNR value of a strong baseline CA-TCC almost remains steady after training, while the SNR value of our method is boosted significantly. See Appendix B.6 for detailed illustrations. The analyses verify that DECL can indeed mitigate noise in representation learning.

5 RelatedWork

5.1 Time Series Self-supervised Learning

To alleviate the reliance on numerous annotated training data for supervised learning, an increasing number of researches focus on learning representations of time series in a self-supervised manner. Time series SSL methods roughly fall into three categories [Zhang et al., 2023a]: (1) contrastive-based methods [Meng et al., 2023b][Ma et al., 2023][Meng et al., 2023a], which are characterized by constructing positive and negative samples for contrastive learning, (2) generative-based methods [Zerveas et al., 2021][Li et al., 2022], which minimize the reconstruction error between raw data and the generated counterparts for model learning, (3) adversarial-based methods [Seyfi et al., 2022], which usually leverage generator and discriminator for adversarial learning. Among them, the contrastive-based methods are the dominant ones. The key steps of contrastive learning involve building similar (positive) and dissimilar (negative) pairs of data samples and mapping the representations of positive pairs nearby while mapping those of negative pairs farther apart. The contrastive-based methods can be further divided into sampling-based [Y`eche et al., 2021][Tonekaboni et al., 2021], augmentation-based [Yue et al., 2022][Yang and Hong, 2022], and prediction-based [Oord et al., 2018][Deldari et al., 2021]. A recent method called TS-CoT [Zhang et al., 2023b] assumes that complementary information from different views can be used to mitigate data noise. Different from it, our method leverages the power of conventional denoising methods to guide representation learning. More details are shown in Appendix D.1.

5.2 Time Series Denoising

Existing time series denoising methods can be divided into two categories: conventional methods and learning-based methods. The conventional methods include empirical mode decomposition [Huang et al., 1998], wavelet filtering [Chaovalit et al., 2011], sparse decomposition-based, Bayesian filtering [Barber et al., 2011], and hybrid method (e.g., waveletbased ICA [Castellanos and Makarov, 2006]) for noise mitigation. The advantages of conventional methods lie in the great denoising effects if properly adopted to match the noise type. However, selecting suitable denoising methods requires prior knowledge or trial-and-error. To avoid human efforts, many learning-based methods that can mitigate the effect of noise have been proposed in recent years. Based on network architecture, these methods include wavelet neural networks, RNN-based [Zhang et al., 2023d][Yoon et al., 2022], and auto-encoders [Zheng et al., 2022]. Differing from the previous efforts, our work leverages the conventional denoising methods to guide mitigating noise in learning, which combines the advantages of the two method categories.

6 Conclusion

In this work, we investigate the problem of mitigating the effect of data noise for time series SSL. Accordingly, we propose an end-to-end method called DEnoising-aware Contrastive Learning (DECL) for noise elimination in representation learning. It automatically selects suitable denoising methods for every sample to guide learning and performs a customized contrastive learning toward obtaining noise-free representations. Extensive empirical results verify the effectiveness of our method. Additionally, we perform comprehensive analyses to verify our claims, such as the distribution visualization of reconstruction errors and the denoising effect visualization of the selected methods. Future works can explore (i) how to automatically determine suitable hyperparameters of the denoising methods; (ii) examine the effectiveness of the method for more downstream tasks, e.g., forecasting and anomaly detection [Lai et al., 2021].

Ethical Statement

There are no ethical issues.

Acknowledgments

This research was supported by the grant of DaSAIL project P0030970 funded by PolyU (UGC).