ImputeINR: Time Series Imputation via Implicit Neural Representations for Disease Diagnosis with Missing Data

1 Introduction

Healthcare data inherently exhibit a temporal structure, as they are often collected sequentially over time in the form of physiological signals, electronic health records, and clinical monitoring data, making time series analysis a fundamental approach for understanding and predicting healthrelated outcomes [Schaffer et al., 2021; Wang et al., 2018; Li et al., 2023a; Li et al., 2023b]. However, real-world healthcare data are frequently compromised by a substantial amount of missing values, which arise from various sources, such as sensor malfunctions, transmission errors, or irregular reporting intervals. The presence of missing values poses a significant challenge to disease diagnosis, as incomplete or corrupted data can distort model training and lead to unreliable predictions. Consequently, the imputation of missing data becomes a critical step in disease diagnosis. By filling in the gaps, imputation enables the restoration of a complete dataset, allowing diagnostic models to perform without the biases or inaccuracies that would otherwise result frommissing information.

∗Corresponding authors.

However, most existing imputation methods do not highlight the cases of extremely sparse observed values. These works assume that the proportion of masked values requiring imputation does not exceed 50%, which means that these methods still require a certain amount of known information. But in real-world healthcare data, the proportion of missing values is likely to be even higher. For example, the MIMIC-III dataset [Johnson et al., 2018], a public healthrelated database, contains 63.15% missing values. The PhysioNet 2012 and 2019 challenges [Silva et al., 2012; Reyna et al., 2019], which involved intensive care unit (ICU) patient data, reported a missing data rate of 79.67%, highlighting the prevalence of high missing rates in real-world healthcare datasets. Although some recent studies try to address imputation tasks in sparse scenarios [Alcaraz and Strodthoff, 2022; Tashiro et al., 2021], most work relies on diffusion-based methods, which involve the simulation of high-dimensional stochastic processes and multi-step iterative procedures. Consequently, these approaches are typically time-consuming and require a large number of training iterations, and their limited effectiveness in achieving accurate imputation can significantly impair the performance of downstream disease diagnosis tasks. How to perform imputation in extremely sparse scenarios remains challenging.

Recently, implicit neural representation (INR) has emerged as an effective method for continuously encoding diverse signals [Liu et al., 2023b; Molaei et al., 2023]. As shown in Figure 1, it learns continuous functions from discrete data points, mapping coordinates to signal values. By representing complex structures in a compact form, INR is not coupled to sampling frequency anymore, which allows for multi-sampling frequency inputs enabling effective feature extraction even with absent observed samples. Additionally, as a continuous function, INR has infinite sampling frequency, which means it can be queried at any coordinate. This capacity for infinite sampling frequency interpolation sets it apart from other imputation methods, making it a promising approach for finegrained imputation.

In this paper, we propose a novel time series imputation approach, named ImputeINR, which achieves effective imputation even in extremely sparse scenarios and the imputed data can further enhance the performance of downstream disease diagnosis tasks. Generally, we learn an INR continuous function for the target time series data and enable fine-grained interpolation with infinite sampling frequency. The parameters of the INR function are predicted by a transformerbased feed-forward network conditioned with sparsely observed values. More specifically, an adaptive group-based form of the INR function is proposed for capturing complex temporal patterns and cross-channel correlation features. It is a multilayer perceptron (MLP) composed of global layers and group layers. The former focuses on correlation information across all channels, while the latter emphasizes correlation information among variables within a single group. We observe that INR exhibits the strongest representational capacity when partitioning variables with similar distributions into the same group. Therefore, variable clustering is applied to determine the variable partition. Additionally, a multi-scale feature extraction module is incorporated to capture patterns at various temporal scales, achieving better fine-grained imputation. Experimentally, ImputeINR achieves state-of-theart performance on eight imputation benchmarks with various ratios of masked values. Moreover, we demonstrate that the imputed data with ImputeINR, compared to other methods, significantly improves the performance of downstream disease diagnosis tasks. The major contributions of this paper are summarized as follows:

• We propose ImputeINR, which learns INR continuous function to represent the continuous time series data. It achieves effective imputation in extremely sparse scenarios and the imputed data can improve the performance of downstream disease diagnosis tasks.
• We design an adaptive group-based form of the INR continuous function to effectively capture intricate temporal patterns and cross-channel correlation features.
• We apply variable clustering to determine the variable partition, allowing our group-based architecture to learn correlation information across all variables and among variables with similar distributions.
• Extensive experiments show that ImputeINR outperforms other baselines on eight datasets under five ratios of masked values. We also demonstrate that using ImputeINR for healthcare data imputation significantly enhances disease diagnosis performance.

2 RelatedWork

2.1 Time Series Imputation

The earliest time series imputation methods are based on the statistical properties of the data, using mean/median values or statistical models to fill in missing values, such as SimpleMean/SimpleMedian [Fung, 2006] and ARIMA [AfrifaYamoah et al., 2020]. Then, machine learning methods learn data patterns, showing greater adaptability and accuracy. Prominent works of these approaches include KNNI [Altman, 1992] and MICE [Van Buuren and Groothuis-Oudshoorn, 2011]. Although these methods are easy to interpret, their limitations lie in capturing the complex temporal and variable information inherent in time series data. Recently, there has been widespread interest in using deep models to capture complex temporal patterns for imputation of missing values, due to their powerful representation capabilities. Common architectures include RNN-based methods (e.g., M-RNN [Yoon et al., 2018], NRTSI [Shan et al., 2023], and BRITS [Cao et al., 2018]) , CNN-based methods (e.g., TimesNet [Wu et al., 2023]), MLP-based methods (e.g., DLinear [Zeng et al., 2023], TimeMixer [Wang et al., 2024]), transformer-based methods (e.g., SAITS [Du et al., 2023], FPT [Zhou et al., 2023], iTransformer [Liu et al., 2024], ImputeFormer [Nie et al., 2024]), diffusion-based methods (e.g., CSDI [Tashiro et al., 2021], SSSD [Alcaraz and Strodthoff, 2022]).

2.2 Implicit Neural Representations

INR uses neural networks to model signals as continuous functions rather than explicitly representing them as discrete points. It captures complex high-dimensional patterns in data by learning a continuous mapping from input coordinates to output values. Various scenarios have seen successful applications, such as 2D image generation [Saragadam et al., 2022; Liu et al., 2023a; Zhang et al., 2024], 3D scene reconstruction [Yin et al., 2022; Liu et al., 2023b; Yang et al., 2024], and video representations [Mai and Liu, 2022; Guo et al., 2025]. Since INR learns a continuous function, it is not coupled to the resolution, which implies that the memory needed to parameterize the signal does not depend on spatial resolution but rather increases with the complexity of the underlying signal. Also, INR has infinite resolution, which means it can be sampled at an arbitrary sampling frequency. Therefore, we leverage this characteristic of INR to perform time series imputation tasks. Sampling from the continuous function of INR enables fine-grained imputation even with extremely sparse observed data. To learn the INR for target signal, there are mainly two typical strategies: gradient-based meta-learning methods [Lee et al., 2021; Liu et al., 2023a] and feed-forward hyper-network prediction methods [Chen and Wang, 2022; Zhang et al., 2024]. In this work, we use a transformer-based feed-forward method to predict the INR for time series data since it can be easily adopted to an end-to-end imputation framework.

2.3 Implicit Neural Representations on Time Series Data

Several works have explored the use of INR to model time series data. INRAD [Jeong and Shin, 2022] leverages INR

to reconstruct time series data for anomaly detection. TSINR [Li et al., 2024] takes advantage of the spectral bias property of INR, prioritizing low-frequency signals and exhibiting reduced performance on high-frequency anomalous data to identify anomalies. However, these methods are specifically designed for anomaly detection tasks and are not suitable for imputation tasks. Currently, only a few works have attempted to leverage INR for time series imputation tasks, but each of these approaches has its own limitations. For example, HyperTime [Fons et al., 2022] utilizes INR to learn a compressed latent representation to capture the underlying patterns in time series for imputation and generation. However, it uses a permutation-invariant set encoder to extract features, leading to an insufficient representation of the underlying patterns in the time series. In addition, TimeFlow [Naour et al., 2023] utilizes continuous-time-dependent modeling and INR enhanced by a meta-learning-driven modulation mechanism for imputation and forecasting. However, treats each variable as an individual sample, thereby ignoring crucial intervariable correlations. Moreover, it fits the entire time series at once becoming computationally inefficient, particularly when dealing with long time series.

3 Methodology

3.1 ProblemFormulation

Imputation with Missing Data

Denote time series data with N variables and T timestamps as X = {x1, x2, . . . , xN} ∈ RN×T . The time series data X is incomplete and the mask rate is r ∈ [0, 1]. The corresponding binary mask matrix can be defined as M = {mn,t} ∈ {0, 1}N×T , where mn,t = 1 if xn,t is observed, and mn,t = 0 if xn,t is missing. The imputation task is to predict the missing values Xmiss such that the predicted values ˆX satisfy ˆX = Fθ(X,M), where Fθ mentions the model with parameters θ. The goal is to minimize the reconstruction error between the masked data and the imputed data:

where |Mmiss| is the total number of missing values in X and Xgt is the ground truth.

Disease Diagnosis

Let Ximputed represent the imputed version of X, where the missing values in X are imputed and the non-missing values in X are retained. The task is to utilize Ximputed for disease diagnosis, where a model G(·) is trained to predict disease labels ˆ y ∈ {0, 1} based on Ximputed:

where ˆ y = 0 denotes normal, and ˆ y = 1 indicates the presence of disease.

3.2 Method Overview

The core idea of ImputeINR is to leverage the inherent capability of INR to learn continuous functions, allowing for fine-grained imputation of time series data by querying at arbitrary timestamps. By leveraging this flexibility, ImputeINR can generate smooth and accurate estimates for missing data points, even with extremely sparse observed data. However, since time series data has inherently intricate temporal patterns and multi-variable properties, using a simple MLP as the INR continuous function to fit it is challenging. To address these issues, we design a novel form of INR continuous function specifically for time series data. Following the previous work [Fons et al., 2022; Li et al., 2024], this form includes three components to capture the trend, seasonal, and residual information to deal with the unique temporal patterns. Furthermore, to enhance the ability of ImputeINR to model multi-variable data, we propose an adaptive group-based architecture to learn complicated residual information. Each group focuses on variables with similar distributions. And we use a clustering algorithm to determine the variable partition. To further enhance the imputation capability of ImputeINR, we incorporate a multiscale module to capture information at different scales, improving fine-grained imputation performance.

Figure 2 demonstrates the overall workflow of the proposed method. The masked data is first reordered based on the variable clustering results so that variables with similar distributions are placed adjacent to each other. This is to enable the subsequent representation of variables within the same cluster using the same group-based MLP in the INR continuous function. Then the reordered masked data is standardized and segmented into patches to prepare the data tokens. Simultaneously, we initialize the INR tokens, which are learnable vector parameters. The processed data tokens are input into convolutional layers of different scales to extract multi-scale features. Subsequently, these extracted features and the initialized INR tokens are fed together into the transformer encoder to predict the INR tokens. These INR tokens are essentially the parameters of the INR continuous function. Based on these parameters, the INR continuous function takes the timestamp t as input and predicts the masked values. Finally, the imputed data is then fed into the disease diagnosis model to obtain the diagnostic results.

3.3 Variable Clustering

We adopt a clustering algorithm C to cluster the variables of the time series data X ∈ RN×T based on the similarity matrix S ∈ RN×N , which partitions the variables into K clusters:

where Ck is a subset of the total variable set {x1, x2, . . . , xN} and its cardinality |Ck| denotes the number of variables in this cluster. The objective of the clustering function C is defined as follows:

where S(xi, xj) represents the similarity between variables xi and xj. Then we obtain the permutation matrix P ∈ RN×N :

where π is the permutation vector that orders the variables according to the clusters. Finally, the reordered matrix X′ with columns permuted according to π is given by:

In this reordered matrix X′ , rows (i.e., variables) are grouped according to the clusters.

3.4 Multi-scale Feature Extraction

To further capture features from different scales for finegrained imputation, the reordered data X′ ∈ RN×T is fed to multiple convolutional layers with varying scales. Each convolutional layer l refers to kernel size kl, stride sl, padding pl, and the number of output channels cl. For each output channel i in the lth convolutional layer, the convolution operation can be formulated as:

where W and b denote the weight matrix and bias matrix respectively. Then these features of different scales Φl (X′ ) ∈ Rcl×(T-kl+2pl+1) are concatenated to obtained the multiscale features ˙X ∈ R∑Ll=1 cl×(T ′-kl+2pl+1) . Finally, these features are fed to the transformer encoder together with the initialized INR tokens to predict the INR tokens.

3.5 INRContinuous Function

INR continuous function f maps the timestamp t to time series data:

where X(t) represents the output values of N variables at timestamp t. To effectively capture the complicated temporal patterns and successfully model the multiple variables, we design a novel form of INR continuous function. Following the previous work [Fons et al., 2022; Li et al., 2024], our INR continuous function includes three components to model trend, seasonal, and residual patterns separately. It can be defined as follows:

where t is the input timestamp and f(t) denotes the output (i.e., imputed data). The parameters in the INR continuous function are predicted by the transformer encoder (i.e., INR tokens).

Trend and Seasonal

The trend represents the long-termmovement or direction of the time series data. It is typically smooth and reflects gradual shifts in the level of the time series, free from noise or shortterm fluctuations. Mathematically, it can be modeled as a polynomial function:

where αi denotes the coefficients and m refers to the degree of the polynomial. In addition, the seasonal component focuses on the repeating patterns or cycles in the time series data, representing predictable fluctuations due to seasonality or recurring events. These regular, cyclical, and short-term fluctuations are modeled with a periodic function:

where βi and γi are Fourier coefficients.

Residual: Adaptive Group-based Architecture

The residual component represents the unexplained variation after removing the trend and seasonal effects, often modeled as a stochastic process. It is challenging to capture this complex information. As discussed in Figure 3, we find that regardless of the order of the variables, using a single MLP is not effective in modeling multiple variables from different distributions. However, if variables from the same distribution are represented using the same set of MLP layers, the performance will significantly improve. We define such a set as a group. In addition, the layers in the MLP that extract information across all variables are called global layers, while the layers within groups are referred to as group layers. The number of groups and their outputs are determined by the results of variable clustering, which allows our architecture to adapt to datasets with various characteristics. It is worth noting that when variables with different distributions are in the same group, the representation capability is significantly reduced. This proves the importance of the correlation information between the variables.

Theoretically, for any given timestamp t, we design L1 global layers, L2 group layers, and K groups. K is determined by the results of variable clustering. The global layers are given as follows:

where l1 ∈ [1,L1], h(l1) is the output of the lth 1 global layer, W and b are weight matrix and bias matrix. Then, for group gk, the input is the output of the last global layer:

where l2 ∈ [1,L2], ˆxl2 gk refers to the output of the lth 2 group layer in group gk, W and b are weight matrix and bias matrix. ˆxL2 gk ∈ R|Ck| and |Ck| is the number of variables in the kth cluster. The final output is the concatenation of the outputs from the last group layer of each group:

4 Experiments

4.1 Experimental Setup

Datasets and Baseline Methods

We use eight time series imputation benchmark datasets to validate the performance of ImputeINR, including ETT [Zhou et al., 2021], Weather [Institute, 2020], BAQ [Zhang et al., 2017], IAQ [Repository, 2008], Solar [Laboratory, 2006], Phy2012 [Silva et al., 2012], Phy2019 [Reyna et al., 2019]), and MIMIC3 [Johnson et al., 2018]. Further, we select the Phy2012, Phy2019, and MIMIC3 healthcare datasets to validate the effectiveness of the imputation results produced by ImputeINR for downstream disease diagnosis tasks. To verify the superiority of ImputeINR, we compare our method to nine state-of-the-art imputation methods, including RNN-based methods (BRITS [Cao et al., 2018]), CNN-based methods (TimesNet [Wu et al., 2023]), MLP-based methods (TimeMixer [Wang et al., 2024]), transformer-based methods (SAITS [Du et al., 2023], FPT [Zhou et al., 2023], iTransformer [Liu et al., 2024], ImputeFormer [Nie et al., 2024]), and diffusion-based methods (CSDI [Tashiro et al., 2021], SSSD [Alcaraz and Strodthoff, 2022]).

Experimental Settings

We apply the same data processing techniques and parameter settings. A sliding window approach is used, with a fixed window size of 48 for the Phy2012, Phy2019, and MIMIC3 datasets, and 96 for all other datasets. These settings follow those used in previous work [Wu et al., 2023; Du, 2023]. To evaluate the imputation performance, we use the same masking strategy as previous works [Wu et al., 2023; Zhou et al., 2023], which randomly mask values in Xgt based on the mask rate r. For the imputation results, the multi-scale feature extraction module uses three parallel convolutional layers with kernel sizes of 3,5,7 respectively. The adaptive group-based architecture in the INR continuous function involves one global layer and one group layer within the residual component, with hidden dimensions set to 16. The transformer encoder consists of 6 blocks. We use the agglomerative clustering method to achieve variable clustering since it adopts diverse inputs without the need to pre-specify the number of clusters. For the downstream disease diagnosis task, we employ the default Long Short-Term Memory (LSTM) classifier provided by the official benchmark [Harutyunyan et al., 2019] as the disease diagnosis model.Experiments are performed using the ADAM optimizer [Kingma, 2014] with an initial learning rate of 10-3 . All experiments are conducted on a single 24GB GeForce RTX 3090 GPU.

4.2 Imputation Task

Main Results

We compare our ImputeINR method to nine state-of-the-art imputation methods with five different mask rates r. As shown in Table 1, our ImputeINR achieves the best performance in most conditions in terms of both MSE and MAE metrics. Overall, across all datasets and mask rates, our method achieves an average MSE reduction of 62.0%

compared to the second-best results. These results demonstrate that our proposed ImputeINR can effectively deal with datasets of various sizes. In addition, we observe that the performance of most methods declines as the mask rate r increases. This aligns with our expectations, as fewer samples are captured leading to incomplete information, which increases the difficulty of imputation. However, ImputeINR is still effective even with an extreme mask rate. When 90% of the data is masked, the average MSE of our method is reduced by 68.2% compared to the second-best ones. This indicates that ImputeINR can learn continuous functions from very few data points, achieving fine-grained imputation.

Ablation Studies

In this section, we conduct ablation studies to evaluate the effectiveness of the multi-scale feature extraction block, variable clustering, and adaptive group-based architecture. We set mask rate r to be 50% for the ablation study, as it provides a moderate level of missingness that effectively highlights the impact of each model component while maintaining stability and representativeness in the results. Table 2 presents the imputation results for all conditions. First, the model without any of the three modules exhibits the lowest performance. Building on this, adding any one of the modules will enhance the imputation capability of the model. This individually validates the effectiveness of each of the three modules. Furthermore, the permutation of any two modules will lead to higher performance. Among them, the combination of variable clustering and adaptive group-based architecture yields the best results. This is as expected, since the variable clustering determines the variable partition and its outcomes correspond directly to the number of groups. Therefore, these two modules can support each other, facilitating better representational learning. Finally, the model using all three modules displays the highest imputation performance as each module contributes complementary strengths.

4.3 DownstreamDisease Diagnosis Task

To evaluate the effectiveness of our proposed ImputeINR method in real-world healthcare applications, we conduct disease diagnosis using the imputed data. Specifically, we first apply ImputeINR to impute missing values in healthcare datasets and then use the imputed data for disease diagnosis. As shown in Table 3, the performance is compared against commonly used imputation methods, including zero imputation, mean imputation, and other state-of-the-art imputation techniques. For evaluation, we employ the standard metric of area under the receiver operating characteristic curve (AUROC). The results demonstrate that the imputed data from ImputeINR consistently leads to superior diagnostic performance across all metrics. This indicates that our model not only effectively reconstructs missing values but also preserves critical disease-related patterns, enhancing the downstream diagnostic capability.

4.4 Efficiency Analysis

We evaluate the efficiency of the ImputeINR method, with the results presented in Figure 4. On the x-axis, we plot the running time per iteration, while the y-axis represents the average MSE. The size of each bubble is proportional to the number of model parameters.This figure shows that ImputeINR achieves accurate imputation results while requiring relatively little running time. Moreover, the corresponding bubble size suggests that ImputeINR has a relatively small number of model parameters, further emphasizing its efficiency. These results prove that ImputeINR strikes a favorable balance between computational efficiency and imputation accuracy.

5 Conclusion

In this paper, we propose ImputeINR, which achieves effective imputation in extremely sparse scenarios and the imputed data can further enhance the performance of downstream disease diagnosis tasks. In contrast to existing imputation approaches, ImputeINR leverages the sampling frequencyindependent and infinite-sampling frequency capabilities of INR to achieve fine-grained imputation with sparse data.

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos. 62202422 and 62372408).