MMNet: Missing-Aware and Memory-Enhanced Network for Multivariate Time Series Imputation

1 Introduction

Multivariate time series(MTS) represent a crucial type of data in the real world, providing valuable insights for decisionmaking across various fields [Yi et al., 2023b], including energy [P´erez-Lombard et al., 2008], healthcare [Esteban et al., 2017], weather [Bright et al., 2015], and transportation [Li et al., 2018]. However, in practical scenarios, multivariate time series are typically incomplete due to various factors, such as sensor malfunctions [Miao et al., 2021], unstable system environments [Wu et al., 2022; Wu et al., 2023b], or privacy concerns [Wu et al., 2023c; Wu et al., 2022]. For instance, the average missing rate of public real-world medical dataset PhysioNet exceeds 80%

[Silva et al., 2012], which significantly hinders time series analysis. Consequently, addressing the problem of imputing missing values in incomplete multivariate time series data has become a critical challenge.

Early approaches for MTS imputation typically relied on statistical or machine learning methods, such as mean averaging, ARIMA [Nelson, 1998], and KNN [Peterson, 2009]. While these methods are generally effective in capturing the linear characteristics of time series, they struggle to model the complex and nonlinear temporal dependencies. In recent years, imputation methods based on deep neural networks have achieved remarkable success in both deterministic and probabilistic imputation. These approaches commonly employ autoregressive models to process incomplete MTS data.

Nevertheless, to effectively encode missing data, these methods often require a preliminary step of fully filling in the missing components. Currently, most deep learning methods [Ma et al., 2019; Du et al., 2023; Bansal et al., 2021] primarily initialize missing values with zeros, and a missing mask matrix is used to mark the locations of the missing data. However, this approach will unavoidably introduce noise, complicating the model's ability to differentiate between missing and observed data, while also failing to capture variations in missingness across multiple time series.

Moreover, the inherent non-stationarity and variability of time series data pose significant challenges for accurately imputing historical observations. Although a relationship exists between the observed and missing components, this relationship is complex and further exacerbated in incomplete scenarios. In the presence of missing data, the available information for time series imputation is sparse. Existing methods [Chen et al., 2023; Li et al., 2018; Luo et al., 2019] typically rely on limited local information within the sample for imputation. However, these methods overlook the fact that similarity in time series data is not limited to local samples, but also extends globally, as similar time segments can be identified beyond the sample.

In this paper, we propose a Missing-Aware and MemoryEnhanced NetWork named MMNet for incomplete multivariate time series imputation. Specifically, MMNet introduces a missing-aware embedding (MAE) module, which uses learnable vectors to represent missing components, enabling the model to differentiate between missing and observed data while adapting to varying missing patterns. To capture global patterns in MTS, MMNet introduces a memory-enhanced encoder (MEE) that leverages persistent memory vectors to integrate global prior information while modeling temporal and variable dependencies. Moreover, MMNet employs a multiscale mixing (MSM) architecture for more robust imputation. MSM encodes incomplete multivariate time series data across multiple scales, progressively merging coarse-fine information, thereby integrating both macroscopic and detailed data to improve imputation performance. We conduct extensive evaluations on four real-world MTS datasets. The results show the superior performance of our proposed method by comparison with current SOTA models. Our main contributions are summarized as follows:

• We propose MMNet, a Missing-Aware and MemoryEnhanced Network for multivariate time series imputation. MMNet adaptively learns the missingness in MTS and utilizes a memory mechanism to effectively capture and leverage the global information in the time series.
• We propose a MAE module for incomplete MTS embedding, utilizing learnable vectors to adaptively represent missing components in incomplete MTS, thus avoiding uncontrollable noise.
• We introduce a MEE module that captures temporal and variable dependencies while utilizing persistent memory vectors to model global similarities in time series data.
• Extensive experiments using four real-life MTS datasets demonstrate that, MMNet substantially outperforms the state-of-the-art methods.

2 RelatedWork

Multivariate time series imputation has attracted increasing attention, driven by the growing availability of time series data and its diverse applications. In the early stages, statistical methods were widely employed to address missing values in multivariate time series data. These approaches typically substitute missing values with statistical estimates, such as zero, the mean, or the last observed value, or utilize simple statistical models like ARIMA [Nelson, 1998], ARFIMA [Hamzac¸ebi, 2008], and SARIMA [Hamzac¸ebi, 2008]. However, the reliance on linear assumptions limit their ability to capture the dynamic and nonlinear features of time series data. Recently, various deep learning-based methods have been developed to address the limitations of traditional approaches in time series imputation, which can be broadly categorized into two classes: forecasting-based methods and generating-based methods.

Forecasting-based methods provide consistent imputation results but often overlook the inherent uncertainties in the data. RNN-based methods have become integral to time series analysis, demonstrating their effectiveness in handling missing data, e.g., GRU-D [Chung et al., 2014], M-RNN [Yoon et al., 2019], Brits [Cao et al., 2018]. CNN-based methods have also gained prominence in MTS imputation tasks, offering robust performance with TimesNet [Wu et al., 2023a] and ModernTCN [Luo and Wang, 2024] being notable examples. GNN-based methods utilize graph structures to model MTS [Yi et al., 2023a; Zhu et al., 2023], leveraging learned node representations to reconstruct missing values, e.g., GRIN [Cini et al., 2022], SPIN [Marisca et al., 2022]. Transformer-based methods [Wen et al., 2023] are also used to model time series, leveraging self-attention mechanisms to effectively capture complex dependencies, e.g., CDSA [Ma et al., 2019], DeepMVI [Bansal et al., 2021], SAITS [Du et al., 2023], OFA [Zhou et al., 2023].

Generating-based methods utilize generative models to simulate and generate missing data. These methods are notable for their ability to produce diverse outputs for missing observations, quantifying computational uncertainty as probability distributions by learning the distribution from the observed values. VAE-based methods typically employ an encoder-decoder structure to approximate the true data distribution, optimizing the evidence lower bound (ELBO) to achieve this goal, e.g., GP-VAE [Fortuin et al., 2020], V-RIN [Mulyadi et al., 2022], supnotMIWAE [Kim et al., 2023]. GAN-based methods leverage a min-max game between the generator and discriminator: the generator simulates the real data distribution, while the discriminator distinguishes between real and generated data, e.g., GRUI-GAN [Luo et al., 2018], E2GAN [Luo et al., 2019], SSGAN [Miao et al., 2021]. Diffusion model use Markov chain process to capture complex data distribution, e.g., CSDI [Tashiro et al., 2021], SSSD [Alcaraz and Strodthoff, 2022], CSBI [Chen et al., 2023], Score-CDM [Zhang et al., 2024].

However, current deep learning methods exhibit certain limitations. A major drawback is that they often require the initialization of missing values, which may degrade the performance of imputation precision. In addition, they frequently fail to effectively capture global patterns in time series data and incorporate prior knowledge.

3 ProblemDefinition

Given a MTS dataset D = {X1, · · · ,XN}, each sample X with d variables and L time steps can be formally denoted as X = (x1, · · · , xd)⊤ = (x:1, · · · , x:L) ∈ Rd×L, where xi can be represented as (xi1, · · · , xiL). Specifically, xij is the value of the i-th variable at the j-th time step in X, and this value may be missing in the incomplete multivariate time series dataset.

Incomplete multivariate time series. To encode the missing information in each sample, a mask matrix M = (m1, · · · ,md)⊤ = (m:1, · · · ,m:L) ∈ {0, 1}d×L is introduced to indicate whether a value in X is missing. Here, mi = (mi1, · · · ,miL), and mij equals 0 or 1, indicating that xij is either missing or observable.

Multivariate time series imputation. Given an incomplete multivariate time series sample X, the goal of incomplete multivariate time series imputation is to construct an imputation model Mθ parameterized by θ, which can accurately estimates the missing values in the data. The imputed time series ˆX can be defined as:

where ⊙ denotes element-wise multiplication, and ¯X = Mθ(X) is the reconstructed time series.

4 Methodology

In this section, we present the proposed model, MMNet, with its architecture depicted in Figure 1. The model consists of two key components: the missing-aware embedding (MAE) module and the multi-scale mixing (MSM) module, both specifically designed for incomplete MTS. First, the incomplete multivariate time series data are processed through the MAE module to generate an incomplete time series embedding. This embedding is then passed to the MSMmodule, where each scale's encoder incorporates a memory-enhanced encoder (MEE) to effectively capture both variable and temporal dependencies via a memory-enhanced mechanism. Finally, a coarse-to-fine mixing approach integrates the encodings across multiple scales, and a linear layer is applied to the mixed encoding to impute the missing values.

4.1 Missing-Aware Embedding

The missing-aware embedding (MAE) module of MMNet aims to project incomplete MTS into hidden representations, allowing the model to explicitly distinguish between missing and observed values. To achieve this, it first normalizes the data to mitigate the impact of non-stationarity. Subsequently, the self-learning missing embedding is introduced to adaptively encode the normalized incomplete MTS.

Incomplete Time Series Normalization. To address the impact of inherent non-stationarity in time series on the model, we first normalize the input incomplete time series sample X using Z-score scaling prior to data embedding [Patro and Sahu, 2015]. This normalization helps mitigate the distribution shift effects within incomplete multivariate time series data. Specifically, for each incomplete time series xi ∈ X and its corresponding mask vector mi, the model first computes the mean µi and standard deviation σi based on the observed data:

Next, the normalized time series ¨xi ∈ ¨X can be defined as: ¨xi = ( xi1-µi σi , · · · , xiL-µi σi ). To preserve the nonstationarity of the time series, the model applies inverse normalization on the output data. Specifically, the predicted time series is restored to its original scale using the stored mean and standard deviation.

Self-Learning Missing Embedding. Subsequently, to prevent pre-filled values from interfering with the model and introducing noise, we propose a self-learning missing embedding method to adaptively represent the missing values in multivariate time series. Regarding the observed data, the model employs a trainable linear projection to generate its embedded representation. Furthermore, for each time series xi, a unique missing embedding vector [MASK]i is learned to represent the missing components in the sequence. In this way, the model can more effectively represent and distinguish the missing states of different time series. Additionally, a positional encoding is utilized to capture the temporal order, with the specific formula provided below:

where Wo ∈ RD×1 represents the learnable projection matrix, Wpos j ∈ RD represents the absolute positional embedding for position j, and [MASK]i ∈ RD denotes the learnable missing embedding vector for the i-th time series. Finally, the model produces a 2D vector array H = (h1, · · · , hd)⊤ = (h:1, · · · , h:L) ∈ Rd×L×D , where hij represents the embedded observable or missing data for the i-th time series at the j-th timestamp.

4.2 Memory-Enhanced Encoder

Given an incomplete multivariate time series as input, selfattention operations, commonly used in Transformers, are capable of capturing both temporal and inter-variable dependencies, thereby enabling more effective imputation. Notably, the basic self-attention mechanism focuses primarily on local relationships within the input sample, as the attention weights are determined solely on the pairwise similarities between the linear projections of the input. Nevertheless, the similarity in time series lies not only within the sample itself but also is reflected in the global similarity across samples.

To fully exploit the global similarity inherent in time series, we introduce a memory-enhanced attention (MEA), which encodes global prior knowledge of time series by incorporating additional learnable memory vectors into the attention computation. It can be formally defined as:

where Wq,Wk,Wv are learnable attention weights in the attention mechanism, and Mem ∈ Rnm×D is a matrix containing nm learnable memory vectors. The symbol ∥ denotes concatenation. Intuitively, by adding additional learnable memory vectors, the attention mechanism becomes capable of retrieving and capturing global relationships beyond the individual sample.

Based on this, we propose a memory-enhanced encoder (MEE) to capture both temporal and inter-variable relationships in MTS. The encoder is composed of multiple stacked two-stage memory-enhanced attention layers, each receiving a two-dimensional vector matrix Z = (Z1, · · · ,Zd)⊤ = (Z:1, · · · ,Z:l) ∈ Rd×l×D as input, where l and d represent the dimension of the time and variable, respectively.

To capture the temporal dependencies of each time series, the model applies the memory-enhanced attention layer to the time dimension of each time series Zi (1 ≤ i ≤ d), i.e.

where LN denotes layer normalization, MLP(·) refers to a multi-layer perceptron, and MEA(·) denotes the memoryenhanced attention layer. The output of the above process is denoted as ¯Z. Next, to capture the inter-variable relationships, the model applies the memory-enhanced attention mechanism along the variable dimension of the output ¯Z:i (1 ≤ i ≤ l), as formulated in the following equations:

Then the output ˆZ fully integrates information from both inter-variable and temporal dimension. Additionally, by leveraging the memory-enhanced mechanism, the model can implicitly capture global similarities in the time series, resulting in more informative representations.

4.3 Multi-scale Mixing

Time series exhibit distinct characteristics at different scales, making multi-scale modeling essential for capturing their inherent patterns. In MTS imputation, coarse scales capture general trends, while fine scales reconstruct finer details.

Building on the memory-enhanced encoder, MMNet utilizes a multi-scale mixing(MSM) architecture to extract and integrate multi-scale features from incomplete multivariate time series. The encoder at each scale transforms the input into a hidden representation, capturing features specific to that scale. For an architecture with S scales, the input and output of the encoder at each scale can be defined as:

where DownSample(·) denotes the downsampling layer implemented by using a convolution operation in the time dimension (kernel=3, stride=2, and padding=1). MEE(·) denotes the memory-enhanced encoder. Zin,i and Zout,i respectively represent the input and output of the encoder at the i-th scale, and H denotes the output of the MAE. We can obtain a multi-scale encoding set [Zout,1 ,Zout,2 , · · · ,Zout,S ].

Considering that coarse-scale time series provide clearer macro-level information compared to fine-scale time series, the model adopts a coarse-to-fine mixing strategy to combine the encoding of S scales, leveraging macro knowledge from coarser scale to guide the imputation modeling at finer scale:

where UpSample(·) is the upsampling layer implemented by a deconvolution operation in the time dimension.

Finally, a linear head is applied to the mixed encoding Zout,1 to obtain the reconstructed multivariate time series. By combining the observable values of the original sample with the reconstructed series, the imputed time series is obtained and formally expressed as:

where Wz ∈ R1×D is a learnable linear projection matrix, ⊙ denotes element-wise multiplication, X and M represent the input sample data and its missing matrix, respectively. ¯X ∈ Rd×L and ˆX ∈ Rd×L denote the reconstructed and the imputed time series, respectively.

4.4 Overview Training Process

To train the imputation model more effectively on incomplete multivariate time series, we employs multi-task training optimization referring to SAITS [Du et al., 2023].

In general, the objective of MMNet contains two types of loss: i) masked reconstruction loss, which aims to reconstruct the masked values within the incomplete historical multivariate time series. and ii) observed reconstruction loss, which focuses on accurately reconstructing the observed values. Here, masked reconstruction loss encourages the model to impute missing values accurately, while observed reconstruction loss ensures the model aligns with the distribution of observed data. The final objective can be formalized as:

where || · ||2 represents the L2 norm function, ⊙ denotes element-wise multiplication, and λ is a hyper-parameter.

Specifically, during model training, we randomly mask γ percent of the time series. To encode the artificial masking information, an artificial mask matrix ¯M ∈ Rd×L is used, where ¯mij equals 0 (resp. 1) means xij is unmasked (resp. masked). This objective function is used to train the multivariate time series imputation model in an unsupervised manner. Eventually, the MMNet is trained by minimizing L to obtain the optimal imputation model ¯Mθ. The training pipeline is shown in Algorithm 1.

Algorithm 1: MMNet Algorithm Flow

Input: Incomplete multivariate time series X, missing mask matrixM, number of scales S, training epochs epochs Output: Optimized time series imputation model Mθ 1: for epoch = 1 to epochs do 2: Mask the observable values in Xwith a ratio γ 3: Normalize the masked data 4: obtain the incomplete data embedding H /* Multiscale Encoding */ 5: Zin,1 =H 6: for s = 1 to S do 7: Zout,s = MEE(Zin,s ) 8: Zin,s+1 = DownSample(Zout,s ) /* Multiscale Mixing */ 9: for s = S-1 to 1 do 10: Zout,s = Zout,s + Upsample(Zout,s+1 ) 11: Obtain the reconstructed sequence ¯X and the imputed sequence ˆX using Equation 9 12: Calculate the training loss using Equation 10 13: Update parameters θ using the ADAM optimizer with ∇θL.

14: return Optimized imputation modelMθ

5 Experiment

In this section, we evaluate the performance of MMNet and eight state-of-the-art methods for multivariate time series imputation . All approaches were implemented in Python. The experiments were conducted on an Intel Core 2.80GHz server with TITAN Xp 12GiB (GPU) and 192GB RAM, running Ubuntu 18.04 system.

5.1 Experiment Setup

Dataset. (i) PhysioNet 2012 Mortality Prediction Challenge Dataset(PhysioNet) [Silva et al., 2012] contains 12,000 multivariate clinical time series samples with 37 measurements collected from ICU (Intensive Care Unit) patients. Each sample represents data recorded during the first 48 hours following a patient's admission to the ICU. The dataset contains 80.67% missing values. (ii) Beijing Multi-site Air Quality Dataset(Air-Quality) [Zhang et al., 2017] includes hourly air pollutant data from 12 monitoring stations in Beijing, collected between March 1, 2013, and February 28, 2017 (48 months). For each station, 11 continuous time series variables

(e.g., PM2.5, PM10, SO2) were measured. The dataset contains 1.6%missing values. (iii) Electric Transformer Temperature Dataset(ETT) [Zhou et al., 2021] records hourly electricity consumption from 370 clients across two regions in China. Each data sample includes 7 features, including temperature and 6 different types of external power load characteristics. This dataset consists of four subsets (ETTh1, ETTh2, ETTm1, ETTm2), with the ETTh1 subset used in our experiments. (iv) Italy Air Quality Dataset (Italy-Air) [De Vito et al., 2008] provides hourly average response values from five metal oxide sensors, alongside hourly average pollutant concentration data from certified analyzers. The train/val/test splits for all four datasets follow the same setup as in [Du et al., 2024]. The sequence length for each sample across all four datasets is 48, 24, 48, and 24, respectively.

Baselines. In the experiments, we compare MMNet with eight baseline time series imputation methods. These include one static imputation method (Last) and seven advanced deep learning algorithms: SAITS [Du et al., 2023], Brits [Cao et al., 2018], GPVAE [Fortuin et al., 2020], CSDI [Tashiro et al., 2021], TimesNet [Wu et al., 2023a], Trans[Vaswani et al., 2017] and ModernTCN [Luo and Wang, 2024].

Evaluation metrics. We apply the mean absolute error (MAE) [Chai and Draxler, 2014] and mean square error (MSE) [Jeffery et al., 2006] to evaluate the effectiveness of imputation models. Smaller metric values indicate better imputation performance. For datasets without missing values (ETT and Italy-Air), 50% of the observed values are randomly removed before model training. To compute the evaluation metrics, 20% of the observed values in both the validation and test datasets are randomly masked, and these masked values are treated as ground truth to assess imputation performance. In downstream experiment, classification accuracy is evaluated using ROC-AUC, PR-AUC, and F1-score, with higher values reflecting better performance.

Implementation details. All experiments were implemented using PyTorch [Paszke et al., 2019]. To ensure a fair comparison, the number of training epochs was uniformly set to 500 across all models, with an early stopping strategy employed using a patience value of 10. Additionally, hyperparameter optimization was performed on key parameters of the deep learning-based imputation algorithms. For MMNet, to balance efficiency and accuracy, the number of scales S is set to 3, the number of layers in the encoder at each scale is set to 2, the number of memory vectors nm is set to 12, the loss weight factor λ is set to 1, the artificial masking rate γ for self-supervised training is set to 20%, and the hidden layer dimension is set to 64. To ensure the reliability of the results, all reported metrics represent the average performance across five runs with different random seeds.

Missing patterns. In the experiments, we inspect six missing mechanisms simulated using PyGrinder 1 . The foundational classification scheme proposed by Rubin [Strawderman, 1989] delineates missing data mechanisms into three specific categories: Missing Completely At Random (MCAR), Missing At Random (MAR), and Missing Not At Random (MNAR). Despite this, in real-world scenarios, missing values in time series often exhibit continuity, where a missing point is typically followed by a sequence of consecutive missing points, termed as "block" missing. To account for this characteristic, we extend the three fundamental missing mechanisms by incorporating block missing, resulting in six distinct missing mechanisms: MCAR, MCAR B, MAR, MAR B, MNAR and MNAR B. To ensure fairness, the missing rate Rm, representing the proportion of dropped values in the test, validation, or training data, is set to 50% by default for all generated datasets under various missing mechanisms.

1 https://github.com/WenjieDu/PyGrinder

5.2 Comparison Study

Table 1 reports the experimental results of multivariate time series imputation methods over four datasets. The bestperforming results are highlighted in bold, while the secondbest results are underlined. It can be observed that MMNet consistently outperforms all baseline methods across all datasets and metrics. Specifically, the imputation performance of MMNet, measured by MSE and MAE, is on average 26.87% and 28.62% better than the best baseline method CSDI, and it even increases up to 58.79% and 59.52% on the Italy-Air dataset. This is because MMNet adopts an effective imputation model with the missing-aware embedding and the memory-enhanced encoder, while improving the imputation accuracy by incorporating a multi-scale mixing strategy. Additionally, we observe that among the baseline methods, static imputation algorithms (Last) generally perform worse than most deep learning-based models. Among the deep learningbased approaches, the diffusion-based method CSDI achieves the best performance on most datasets, whereas the VAEbased model GPVAE performs significantly poorer compared to other methods. Therefore, the Last impuation and GPVAE imputation methods will be excluded from the subsequent parts of the experiments.

5.3 DownstreamResult

In this experiment, we evaluate the performance of downstream classification analysis using imputed datasets, which indirectly reflects the performance of each imputation method. The classification task is performed on the PhysioNet dataset, where each sample is labeled to indicate whether the patient has died. Specifically, the imputation methods are first employed to infer missing values in original incomplete datasets. The imputed datasets are then used to train a simple RNN classifier optimized with cross-entropy loss. All hyperparameters are set as follows: the learning rate is set to 0.005, the hidden state dimension is set to 128, the number of training epochs is set to 100, and early stopping is employed with a patience of 20 epochs.

The experimental results are shown in Table 2. It can be observed that datasets with better imputation results tend to exhibit superior performance in downstream classification tasks. Notably, the classifier trained on the dataset imputed by MMNet achieved the best results across all evaluation metrics. Compared to the suboptimal performance, it improved by 0.42%, 1.09%, and 0.89% in PR-AUC, ROC-AUC, and F1-Score, respectively. These findings further highlight that MMNet provides the best imputation quality among all methods and that accurate imputation can improve the performance of downstream classification tasks on incomplete multivariate time series datasets.

5.4 Parameter Evaluation

Effect of Missing Rate. When varying the missing rate Rm (i.e., the proportion of additional observed values in multivariate time series data are dropped) from 10% to 90%, the corresponding experimental results, measured by MAE, are depicted in Figure 2. As observed, the accuracy of all imputation algorithms consistently declines as the missing rate increases, reflected by a continuous rise in MAE. This is because higher missing rates reduce the amount of available information for inference, thereby diminishing the effectiveness of imputation algorithms. Among these algorithms, MMNet consistently outperforms all others in every case and maintains more stable accuracy as the missing rate increases. In other words, MMNet is more robust with the increasing missing rate Rm than others. The underlying reason is that MMNet incorporates a memory-enhanced encoder, enabling it to capture global information beyond the sample, thereby better addressing data sparsity issue.

Effect of Missing Mechnisam. In this set of experiments, we evaluate the effect of different missing mechanisms (i.e., MCAR, MAR, MNAR, MCAR B, MAR B, and MNAR B) on the imputation algorithm performance. The results, measured by MAE, over the ETT dataset are shown in Figure 3. The results indicate that the performance of imputation algorithms is relatively insensitive to different missing mechanisms. Furthermore, "block" missing patterns (i.e., MCAR B, MAR B, and MCAR B) leads to a further decrease in imputation performance compared to the "value" missing patterns (i.e., MCAR, MAR, and MCAR). This is because the "block" pattern hinders the model's ability to infer missing points from adjacent ones, while also lead to more substantial information loss over extended periods, thereby affecting the model's imputation performance. Additionally, as expected, it is evident that MMNet consistently attains the optimal imputation performance in every scenario, further validating its robustness. Specifically, MMNet surpasses the best baseline CSDI with an average improvement of 24.81%, and even increases up to 33.33% on the MCAR missing mechanism.

Effects of Scales. We investigate the impact of the number of scales on the model's imputation performance, with the results presented in Table 4. The results clearly show that as the number of scales increases, the imputation performance improves overall, further validating the importance of integrating information from multiple scales to enhance accuracy.

5.5 Ablation Study

We investigate the influence of different components of MMNet on the imputation performance. The corresponding experimental results are shown in Table 4. MMNet-no-MAE

refers to the variant of MMNet without the missing-aware embedding module, where zero-filling is used instead. MMNet-no-MEE refers to the variant of MMNet without using memory-enhanced attention, and MMNet-no-MSM refers to the variant of MMNet without multi-scale mixing. It can be observed that each module in the MMNet model do contribute positively to imputation performance. Specially, the average imputation accuracy (i.e. MSE and MAE) decreases by 24.70%, 11.17%, and 10.12% without the missing-aware embedding, multi-scale mixing and memory-enhanced attention, respectively. Among these, the missing-aware embedding module has the most significant impact on MMNet. This proves that simple pre-filling strategies indeed interfere with the model's ability. The MMNet model, by introducing MAE module, allows the model to better learn the missing distributions across multiple time series while avoiding noise.

6 Conclusion

In this paper, we propose a Missing-Aware and MemoryEnhanced Network, named MMNet, for multivariate time series imputation. Specifically, MMNet employs the missingaware embedding (MAE) module to adaptively represent incomplete multivariate time series data. The memoryenhanced encoder (MEE) is designed to capture both temporal and variable dependencies while modeling the prior knowledge within the time series. Building on these modules, the multi-scale mixing (MSM) architecture is employed to combine information across multiple scales to enhance imputation performance. Extensive evaluations on four realworld datasets demonstrate the effectiveness of the proposed model, highlighting its superiority in addressing incomplete multivariate time series data.

Acknowledgments

This work is partly supported by the National Key R&D Program of China under Grant No. 2024YFF1400900, the National NSFC under Grants No. 62372404, the Leading Goose R&D Program of Zhejiang under Grant No. 2024C01109, and the Fundamental Research Funds for the Central Universities under Grant No. 226-2024-00030. Yangyang Wu is the corresponding author of the work.