VecoCare: Visit Sequences-Clinical Notes Joint Learning for Diagnosis Prediction in Healthcare Data

1 Introduction

With the widespread adoption of electronic healthcare information systems in various healthcare institutions, many deep learning models have been developed to leverage electronic health records (EHR) data for important medical applications [Yang et al., 2017; Ma et al., 2020; Yang et al., 2021; Feng et al., 2021; Ma et al., 2021; Zhang et al., 2021; Zhang et al., 2022; Ma et al., 2022; Xu et al., 2023]. Diagnosis prediction is one of these vitally important healthcare scenarios. It predicts the future diagnoses of patients based on their historical sequences of clinical events, such as the diagnoses, lab tests, medications, etc.

∗Corresponding Author.

Due to the data insufficiency in practical scenarios, making full use of the information contained in various types of data becomes the focus of deep learning model design. From the perspective of the type of EHR data, some try to leverage the information of structured EHR data (e.g., static baseline information, lab test values, historical diagnoses, etc.) to perform the diagnosis prediction [Ma et al., 2017; Choi et al., 2017; Ma et al., 2018; Luo et al., 2020; Tan et al., 2022]. While some works delve into mining the unstructured clinical notes (i.e., texts written by clinical experts with assessments and concerns regarding patients' clinical conditions) [Thapa et al., 2022]. As shown in Figure 1, compared with structured EHR data recording medical events and physiological signals, unstructured clinical notes describe finegrained information about the patients' family history, social history, the doctors' observations and inferences, etc., which are critically complementary to structured data.

However, synthesizing rich information from structured and unstructured EHR data is still marginally studied for diagnosis prediction. In this work, we exploit the potential synergies between structured EHR data and unstructured clinical notes to form a harmonious representation space for accurate diagnosis predictions. Although seemingly straightforward, implementing this intuition will encounter the following challenges:

C1. It is challenging to synthesize two representation spaces due to data heterogeneity. Some works integrate keywords of the clinical notes based on the similarity of medical events and textual clinical notes for diagnosis prediction [Lu et al., 2021]. However, structured data are discrete codes or continuous values that quantitatively describe changes in a patient's health status. In contrast, unstructured clinical notes are mainly clinicians' qualitative descriptions in the natural language form. They are incompatible in data distributions, metric measures, and semantic representation spaces [Bronstein et al., 2010; Roostaiyan et al., 2017]. With typical attention mechanisms, it is challenging to reduce heterogeneous semantic biases across two types of information for the joint representation of optimal diagnosis results.

C2. It is tough to learn robust representations due to the intermingled quality of partial clinical notes. For the sake of differences in recorders or the urgency of patients' conditions, some patients' clinical notes suffer from misspellings, poor grammar, non-standard abbreviations and insufficient information [Burke et al., 2014], leading the model to learn inadequate representations and resulting in sub-optimal performance. Therefore, it is necessary to refer to relevant information of other patients by exploiting the potential synergies and correlations among them to strengthen the utilization of low-quality clinical notes.

C3. It is difficult to include all valuable patients in the reference range only by typical methods such as attention mechanisms. When complementing low-quality clinical notes with other patients' information, a natural intuition is to focus on significant samples through dot-product attention [Vaswani et al., 2017]. In this way, larger weights are assigned to more similar patients (e.g., in Figure 1, patient #173 and patient #302 are both patients with progressively decreasing hypertensive complications). According to clinicians' practical diagnosis experiences, patients with extremely different or even opposite healthcare statuses (e.g., in Figure 1, patient #651 is a patient with a progressive increase in hypertensive complications) are also essential for diagnosis predictions [Unay and Ekin, 2011; Jia et al., 2020], which is ignored by existing approaches.

To cope with these challenges, we put forward VecoCare, a novel visit sequences-clinical notes joint learning approach to fuse them into a representation space seamlessly for accurate diagnosis prediction. Specifically, our main contributions are summarized as follows:

• To solve the challenge C1, we propose two sequential pre-training tasks to bridge heterogeneous semantic biases across structured EHR data and clinical notes. Firstly, VecoCare incorporates a novel Gromov-Wasserstein Dis-
• tance (GWD)-based contrastive learning task to learn a consistent semantic representation space by maximizing the agreement between visit sequence-clinical note pairs. Secondly, VecoCare further reduces heterogeneous semantic biases with an adaptive masked language model task based on a novel global-local fusing encoder, which can adaptively balance the influence of both types of information.
• Addressing challenge C2 and C3, VecoCare employs a dual-channel retrieval mechanism to aggregate important auxiliary information from similar and dissimilar patients for more comprehensive representations.
• We conduct extensive experiments on two real-world EHR datasets, which show that VecoCare outperforms all state-of-the-art models in different evaluation metrics, including approaches that incorporate task-specific external knowledge. Besides, the medical findings discovered by VecoCare are also in accord with medical literature and can provide valuable medical insights or explanations.

2 RelatedWork

Over the past decade, there has been lots of works focusing on diagnosis prediction with deep learning models, with the vast majority of these methods focusing on mining structured data. One category of methods attempts to focus on capturing contextual dependencies between patient visit sequences. For example, T-LSTM [Baytas et al., 2017] handles irregular time intervals by enabling time decay. RETAIN [Choi et al., 2016] proposes a two-level attention mechanism based on recurrent neural networks (RNN). Dipole [Ma et al., 2017] employs a combination of bi-directional RNN and attention mechanisms to predict diagnoses of patients' next visits. LSAN [Ye et al., 2020] and HiTANet [Luo et al., 2020] employ the selfattention mechanism to capture the temporal patterns. Furthermore, T-ContextGGAN [Xu et al., 2022] and Chet [Lu et al., 2022] build a graph structure according to the patient's medical history, and use graph neural networks (GNN) to learn the representation. Another mainstream category tries to focus on leveraging external medical knowledge graphs to improve representation learning. For example, GRAM [Choi et al., 2017] and KAME [Ma et al., 2018] both incorporate the hierarchy of disease codes to enhance learning. CGL [Lu et al., 2021] constructs a graph with both medical knowledge and personal clinical observations, thence employing a collaborative graph learning method to learn the representations. MedPath [Ye et al., 2021] utilizes the personalized knowledge graph to assist prediction. MetaCare++ [Tan et al., 2022] incorporates domain knowledge of hierarchical and syndromic relations between various diseases. Based on the modeling of structured data, some existing works try to aggregate important information from unstructured clinical notes. For example, CGL [Lu et al., 2021] integrates keywords of the clinical notes based on the similarity of medical events and textual clinical notes. However, they marginally investigated the heterogeneous semantic biases, the intermingled quality of partial clinical notes, and the difficulties in including valuable samples.

3 ProblemFormulation

EHR Dataset. In this paper, EHR data consist of structured time-ordered visit sequences and unstructured clinical notes of patients. Let C = {c1, c2, . . . , c|C|, c∗} be the entire set of codes used in an EHR dataset, where |C| is the number of medical codes. Following [Luo et al., 2020], we also denote a special code c∗ to represent the whole patient data. For the i-th patient, the visit sequence is defined as Xi = [xcls i , x1i , x2i , · · · , xTi ], where the t-th visit is denoted by a multi-hot vector xti ∈ {0, 1}|C| . The k-th element of one visit vector is set to 1 if it contains the medical code ck. xcls i denotes the [CLS] token which only contains the special code c∗. As for the clinical notes, let N = {n1, n2, . . . , n|N|} be the dictionary of clinical notes, where |N| is the number of words. Following [Lu et al., 2021], we select the notes OTi from the patient's T-th visit as another input because it already contains a summary of the observations from the previous T visits. For convenience, we drop the superscript T in OTi in the rest of this paper. Oi can be represented as Oi = [ocls i , o1i , o2i , · · · , oMi ], where M is the length of the clinical notes, omi is represented by a binary vector {0, 1}|N| , and ocls i is the [CLS] token.

Diagnosis Prediction. In this paper, our predictive objective is presented as a diagnosis prediction task, which is a multilabel classification problem. Given the previous T visits Xi and the clinical notes Oi of the i-th patient, the task is to predict a binary vector yi ∈ {0, 1}|Y| , which represents the possible diagnoses in the (T + 1)-th visit, where |Y| is the number of diagnoses.

4 Methodology

Figure 2 shows the architecture of VecoCare. It comprises the following sub-modules in a sequential manner: 1) GWDbased Contrastive Learning Module, 2) Adaptive Masked Language Model Module, 3) Dual-channel Retrieval Module.

4.1 GWD-based Contrastive LearningModule

Base Encoder. Given a sparse binary visit vector xti and a sparse binary word vector omi , we first encode them to a relatively dense space using two linear functions as follows:

where Wv ∈ Rdv×(|C|+1) , bv ∈ Rdv , Ws ∈ Rds×(|N|+1) , and bs ∈ Rds are trainable parameters. As a result, the data of each patient can be represented by Vi = [vcls i , v1 i , v2 i , · · · , vT i ] and Si = [scls i , s1i , s2i , · · · , sMi ]. To explicitly capture the global interactions within the visit sequence and to obtain a time-aware contextual representation, we utilize a time-aware Transformer [Luo et al., 2020] f v as the base visit encoder to extract visit contextual features Hi:

For the clinical notes, we adopt a Transformer [Vaswani et al., 2017] f s as the text encoder backbone to encode notes contextual features Ui:

To map two types of contextual representations to a joint embedding space, we then employ two distinct non-linear projection layers (gv and gs) to convert hti and umi into normalized lower-dimensional embeddings ˜hti ∈ Rd and ˜umi ∈ Rd , respectively.

To learn a consistent semantic representation space for alignment and measurement, a natural idea is to maximizing the agreement between true visit sequences-clinical notes pairs versus random pairs via contrastive learning [He et al., 2020]. However, due to the heterogeneity of the data, the different metric-measure spaces present significant challenges when attempting to reliably evaluate the similarity between these two types of contextual representations. The Gromov-Wasserstein Optimal Transport is employed to minimize the transportation cost between two distributions by directly comparing the metric spaces, rather than evaluating samples across these spaces [Peyr´e et al., 2016; M´emoli, 2011]. Essentially, this framework focuses on the distances between pairs of points within each domain and assesses how these distances correspond to those in alternate domains. As a result, we compute the Gromov-Wasserstein distance (GWD) between the two representation distributions and employ this distance as a loss function to further optimize the representation learning. Specifically, let ph and pu represent discrete distributions of two types of contextual representations, where ph = {p0h , p1h , . . . , pTh } and pu = {p0u , p1u , . . . , pMu }, ∑Tj=0 pjh = ∑Mk=0 pku = 1. The GWD between the two discrete distributions ph, pu can be defined as:

where π(ph, pu) denotes all the joint distributions, T represents the transport plan between two types of features and Tjk denotes the amount of mass shifted from pjh to pku . In addition, ˆ c ( ˜hji , ˜uki , ˜hj ′ i , ˜uk′ i ) = ∥∥∥d ( ˜hji , ˜hj ′ i ) -d ( ˜uki , ˜uk′ i )∥∥∥2 is the cost function to measure the distance between different metric measure spaces where the distance d ( ˜hji , ˜hj ′ i ) = exp (- cos ( ˜hji , ˜hj ′ i ) /τg ) of two features is measured based

on the cosine similarity, τg is the temperature parameter.

Based on GWD, for the i-th visit sequences-clinical notes contextual features pair ( ˜Hi, ˜Ui) in a mini-batch, we alternate between treating two distinct feature types as queries and keys to learn the correct pairings. This results in a pair of symmetric, temperature-normalized InfoNCE losses [Oord et al., 2018] that optimize the preservation of mutual information between the authentic pairs in the latent space:

where B is the batch size and τ is the instance-level temperature hyper-parameter. The overall objective of GWD-based contrastive learning is the average of the two losses:

where Tr is the total number of visit sequences-clinical notes pairs.

4.2 Adaptive Masked Language Model Module

After a pre-training phase based on the GWD-based contrastive learning task, VecoCare learns a consistent semantic representation space. To perform deeper feature information aggregation and integration, we propose a novel globallocal fusing encoder to enable the coupling of global (selfattention) and local (convolution) information. Transformers can effectively capture global long-range dependencies. However, on the one hand, there are temporal patterns of disease changes in neighboring visits [Ye et al., 2020], and on the other hand, there are many information-rich local text fragments (e.g., past medical history, social history, etc.) in the clinical notes. The inductive bias of convolutional neural networks (CNN) can help us aggregate local features. Specifically, given the sequence of features ˜z0 = [ ˜Hi; ˜Ui ] encoded by pre-trained base encoders and projection layers, we propose to add an convolution operation on the values when calculating self-attention:

where l = 1, . . . ,L refers to the number of such stacked layers. MHSA refers to the Multi-Head Self-Attention, FFN refers to a feed forward network, and LayerNorm is the layer normalization. Following [Vaswani et al., 2017], ˜zl-1 v is the value projected directly from ˜zl-1 . After L layers, we can obtain the representation ¯zL = [ ¯Hi; ¯Ui ] which captures global and local token-wise interactions.

Then, we propose an adaptive masked language model (AMLM) task which randomly masks word tokens and medical code tokens and predicts the ground-truth labels from the output of the global-local fusing encoder. It integrates the context of other medical code tokens and textual tokens:

where Cmask is the set of masked medical code tokens, Nmask is the set of masked textual tokens, p (ci) and p (ni) denote the probability of predicting the original token. We adopt the same masking strategy and prediction method as BERT [Devlin et al., 2018].

In order to mitigate the risk of our model developing an over-dependence on particular feature types throughout the pre-training phase, we utilize a technique that actively finetunes the loss weights, ensuring an equitable distribution of influence across the various feature types. To evaluate the extent to which the model fits the target, following [Athalye et al., 2018], we define a relative loss:

where L∗(t) and L∗(0) represent the loss values at time t and at the initial time 0, respectively. A smaller relative loss demonstrates a more rapid convergence of the model to the target. Inspired by [Zhao et al., 2022], we dynamically adjust the loss weights with each update in relation to the relative loss to ensure that both Lc and Ln experience a relatively equitable decline throughout the updating procedure:

where w∗(t) is the loss weight at time t, β is the hyperparameter controlling the degree of updating the loss weight, and mw is the momentum coefficient hyper-parameter. The final AMLM loss is calculated as follows:

With the AMLM loss adjustment, VecoCare can further reduces heterogeneous semantic biases.

Token Feature Text Feature

4.3 Dual-channel Retrieval Module

After performing the above two pre-training stages, now we obtain the contextual representations ¯Hi = [ ¯hcls i , ¯h1i , ¯h2i , · · · , ¯hTi ] and ¯Ui = [ ¯ucls i , ¯u1i , ¯u2i , · · · , ¯uMi ] from the pre-trained global-local fusing encoder. In order to simulate the process of a doctor recalling cases related to the current patient to assist clinical analysis in real world, VecoCare maintains a patient key-value memory bank, which consists of the visit sequences representations and the clinical notes representations of patients in the training set. Specifically, we denote the patient memory bank G as a vectorized indexable dictionary as follows:

where Tr is the training set size, G is initialized at the beginning of training and updated at each training epoch. For clarity, we use Gk = [ ¯hcls 1 ; ¯hcls 2 ; · · · ; ¯hcls T r ] ∈ RTr×d to denote the key vectors and Gv = [ ¯ucls 1 ; ¯ucls 2 ; · · · ; ¯ucls T r ] ∈ RTr×d to denote the value vectors.

For patients with limited information, a natural idea is to rely on information from other patients with similar health conditions. Nevertheless, information from other patients with extremely different or even opposite healthcare statuses can also be useful for medical prediction and treatment prognosis in healthcare [Unay and Ekin, 2011; Jia et al., 2020]. We contend that conventional attention mechanisms are inadequate for modeling negative correlations. Specifically, according to [Vaswani et al., 2017], an attention function can be described as mapping a query and a set of key-value pairs to an output. A typical implementation involves calculating the dot products between the query and the keys, followed by applying a softmax function to derive the weights. The limitation is that the softmax function would treat the negative correlation between query and key as inconsequential. (e.g., executing a dot product and softmax operation on two vectors with opposite directions yields the smallest weight). To address this issue, we introduce a dual-channel retrieval mechanism, comprising both positive and negative channels. For the positive channel, we use the traditional dot-produt attention [Vaswani et al., 2017]. For the negative channel, we capture the negative correlation by inverting the values of the dot-produt between the key and query, and output the opposite of resulting weights after the softmax function:

where αpos , αneg ∈ RTr . In order to fully capture relationships between different perspectives, we use the weights obtained from the positive and negative channels separately to obtain the weighted sum of the values:

where ¯up, ¯un ∈ Rd . Finally, we concatenate ¯hcls i , ¯ucls i , ¯up, and ¯un as the output vector r ∈ R4d for the patient: r = ¯hcls i ∥¯ucls i ∥¯up∥¯un. We then use a simple linear layer with a sigmoid activation function on the model output r to calculate the predicted probability y′i . In this case, the cross-entropy loss is applied as the loss function:

where B is the batch size. y′i ∈ [0, 1]|Y| is the predicted probability, and yi ∈ {0, 1}|Y| is the ground truth.

5 Experiments

In this section, we will show the experimental results including performance comparison, ablation studies and analysis to validate the predictive power and interpretability of VecoCare. The source code is available at 1 .

5.1 Experimental Setup

Datasets

• MIMIC-III Dataset We conduct diagnosis prediction on ICU data from the publicly available Medical Information Mart for Intensive Care (MIMIC-III) database [Johnson et al., 2016]. In this study, we select patients with at least two visits. We use the diagnoses of the last visit as labels and incorporate the remaining visits within a 365-day prediction window as features. For the clinical notes, following [Lu et al., 2021], we select the notes from the last visit in input features and filter out notes of type "Discharge summary" for fair prediction. We study an 135-class classification problem that predicts the diseases of a patient by categorizing ICD-9 codes into 135 broader medical groups using the Unified Medical Language System [Ho et al., 2014].
• RWH Dataset. Another dataset we use is an EHR dataset from a real-world hospital. The cleaned dataset consists of 10,408 patients with 458,952 visits. We adopt the same pre-processing method as used for the MIMIC-III dataset.

Both datasets are fully anonymized and carefully sanitized before our access. We screen patients with clinical notes information during their medical visits. For each note, we use the first 512 words, while the rest are cut off for computational efficiency.

Baselines

To compare VecoCare with state-of-the-art models, we select the following models as baselines:

• T-LSTM [Baytas et al., 2017] introduces a time decay mechanism in LSTM.
• Dipole [Ma et al., 2017] uses bidirectional RNNs and three attention mechanisms to predict patient visit information.
• GRAM [Choi et al., 2017] utilizes medical ontologies to derive code representation learning.
• KAME [Ma et al., 2018] predicts patients' future diagnoses based on a knowledge attention mechanism.
• HiTANet [Luo et al., 2020] incorporates time-awareness into the self-attention component.
• CGL [Lu et al., 2021] proposes collaborative graph learning for enhanced utilization of external medical knowledge.

1 https://github.com/xyxpku/VecoCare

• Chet [Lu et al., 2022] attempts to learn on dynamic disease graphs from patient visit history.
• MetaCare++ [Tan et al., 2022] introduces a clinical metalearner to capture temporal relations among patient visits.

For a fair comparison, we take the clinical notes as additional input, and after obtaining the notes contextual representation via the base text encoder of VecoCare, we concatenate it with the patient representation obtained by baseline methods to perform the prediction. Moreover, we carefully tuned the hyper-parameters of the baselines on the validation set using the grid-search strategy to ensure their best performance. We also conduct the following ablation studies:

• VecoCareg- removes the GWD-based contrastive learning module from VecoCare.
• VecoCarea- removes the adaptive masked language model module from VecoCare.
• VecoCared- removes the dual-channel retrieval module from VecoCare. It directly concatenates ¯hcls i and ¯ucls i to perform the final task.

Evaluation Metrics and Strategy

We assess the performance with four widely used evaluation metrics: micro-averaged of the area under ROC curve (mi-AUROC), macro-averaged AUROC (ma-AUROC), micro-averaged of the area under the precision-recall curve (mi-AUPRC) and macro-averaged AUPRC (ma-AUPRC). AUPRC serves as the most informative and the primary evaluation metric when handling highly imbalanced and skewed datasets like the real-world EHR data [Davis and Goadrich, 2006]. The datasets are divided into the training set, validation set, and test set with a proportion of 0.70:0.15:0.15. Both mean and standard deviation of test performance are reported.

5.2 Experimental Results

Table 1 presents the experimental results of the baseline methods as well as VecoCare on the two datasets. The number in () denotes the standard deviation. The results indicate that VecoCare exhibits a notable and persistent advantage over other baseline methods, especially ma-AUPRC, which is the most informative and the primary evaluation metric. We find that VecoCare outperforms CGL which utilizes keywords from the clinical notes to assist diagnosis prediction. It demonstrates the effectiveness of reducing heterogeneous semantic biases and utilizing auxiliary information from important similar and dissimilar patients. Besides, it is worth mentioning that VecoCare significantly outperforms GRAM, KAME, CGL, and MetaCare++ even without any task-specific external knowledge priors used in these methods. It further validates the significance of synthesizing rich information from structured and unstructured EHR data.

We further conduct the ablation studies to examine the design of VecoCare. The superior performance of VecoCare than the VecoCareg- and VecoCarea- verifies that reducing heterogeneous semantic biases can significantly improve model performance. Moreover, VecoCare outperforms VecoCared-, demonstrating that it is also effective to incorporate information from other relevant patients' clinical notes.

5.3 Analysis

Robustness Against Data Insufficiency

To further explore the benefits of VecoCare in situations with limited data availability, we assess its robustness in the face of inadequate training samples. Specifically, we simulate data scarcity by reducing the RWH dataset's training set from 70% to 30%, 15%, and 10%, while maintaining a fixed test set for unbiased comparison. We conduct experiments under these conditions, and the ma-AUPRC (± std.) is plotted in Figure 3. As shown in Figure 3, VecoCare consistently surpasses the chosen baseline models across all settings. Moreover, as the training set size diminishes, the performance of baseline methods declines more rapidly than that of VecoCare, resulting in a wider performance gap. Impressively, when utilizing only 10% of the data for training, VecoCare still attains a ma-AUPRC of 0.2185 on the RWH dataset, significantly outpacing the baseline methods. This highlights the excellent performance and robustness of VecoCare in scenarios with insufficient data.

Case Study for Relevant Patient Information Retrieval

Now in this section we use case studies on the RWH datasets to show the ability of VecoCare to retrieve relevant patient information. In this example, Figure 4 shows the data of a patient diagnosed with hypertension (401.9), chronic ischemic heart disease (414.9), heart failure (428.9), and acute renal failure (584.5) at the to-be-predicted visit. We also show two other patients selected in the training set from the patient memory bank G with the highest attention weights (absolute value) in the positive and negative channels, respectively. Although the three patients have suffered from the same hypertension disease (401.9), the disease processes of patient #231 and patient #485 are different and opposite. Specifically, we can observe that the hypertension complications of the patient from the testing set (i.e., patient #37) and patient #231 are increasing. However, patient #485 experiences a stepwise reduction in hypertensive complications. It demonstrate that VecoCare is adept at concentrating on patients with analogous disease trajectories while simultaneously paying close attention to those with vastly different or even opposing health statuses, which is helpful for the patient's future treatment prognosis. For example, doctors can refer to the treatment procedures in the clinical notes of patient #485 to mitigate the current patient's condition with digoxin.

Case Study for Attention Analysis of the Fusing Encoder

To further illustrate the interpretability and reasonableness of the proposed VecoCare, we conduct case studies to visualize and interpret the attention weights learned by the globallocal fusing encoder. Figure 5 shows the data of two patients diagnosed with diabetes mellitus at the to-be-predicted visit and VecoCare successfully predicts the outcome. The average attention weights of one head calculated by the selfattention module are shown. The ordinates of the two figures are the Query token [CLS] xcls i and the abscissas are the Key word tokens in the clinical notes. The darker boxes mean that the patient's overall representation is more concerned with the word, and vice versa. For convenience, we visualize only the most noticed word tokens. In the first example, we can observe that VecoCare attaches the highest importance to "Nonalcoholic" and "steatohepatitis", aligning with the medical research [Loomba et al., 2012]. As for another example, We can observe that VecoCare pays more attention to "family", "history", and "diabetes". According to medical research [Valdez, 2009], a family history of diabetes is a major risk factor for this disease. Thus, VecoCare can effectively capture important clinical notes context and reflect them in the form of attention weights for reliable explanations.

6 Conclusions and FutureWorks

In this work, we propose a novel visit sequences-clinical notes joint learning method named VecoCare, which fully considers the fusion of clinical notes with structured data for accurate diagnosis prediction. In particular, VecoCare first reduces heterogeneous semantic biases by two novel sequentially executed pre-training tasks. After that, VecoCare utilizes a dual-channel retrieval mechanism to aggregate information from both similar and dissimilar patients, thus learning a more comprehensive representation. Experimental results on two real-world EHR datasets demonstrate the clear advantages of VecoCare over the state-of-the-art baselines even without any task-specific external knowledge priors. In the meanwhile, case studies further demonstrate the robustness and novel interpretability of VecoCare. Besides, the findings are in accord with experts and medical knowledge, which shows it can provide useful insights. In the future, we will explore incorporating longer and richer clinical notes as well as other data types in EHR to bring higher performance improvements.

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No.62172011), the National Natural Science Foundation of China (No.62102007), and the Fundamental Research Funds for the Central Universities.