Patient-Visit-Spanned Hypergraph Learning for EHR-based Diagnosis Prediction

1 Introduction

The widespread adoption of Electronic Health Records (EHR) has fueled deep learning-based predictive modeling. By uncovering meaningful clinical patterns, these models enhance understanding of patients' health states and support key tasks such as early diagnosis [Che and Liu, 2017], medication recommendation [Kim et al., 2025], and risk stratification [Liu et al., 2022], ultimately empowering proactive interventions and improving patient outcomes [Wu et al., 2025].

According to existing studies, sequence-based models [Luo et al., 2020] and graph neural networks (GNN)-based models [Poulain and Beheshti, 2024] are two mainstream approaches for processing EHR. In particular, sequence-based

∗Corresponding author.

• (a) Vanilla graph of EHR
• (b) Hypergraph of EHR

Figure 1: An illustration of EHR for a patient with three clinical visits. (a) A vanilla graph connects pairs of medical codes that cooccur within a visit. (b) A hypergraph represents each visit as a hyperedge that links all medical codes recorded during that visit.

approaches typically represent clinical visits as temporal sequences to learn dependencies across clinical events [Chen et al., 2025]. However, they often overlook complex interactions among heterogeneous medical codes, including diagnoses, medications, and procedures. In contrast, GNN-based models represent EHR as graphs, where medical codes cooccurring within a clinical visit serve as nodes and edges represent their co-occurrence relationships [Chan et al., 2024], as illustrated in Figure 1(a). Although these models can capture interactions between medical codes, they essentially focus on pairwise relationships, which hinders sufficient exploration of the high-order code interactions.

Hypergraphs [Feng et al., 2019] have been proposed as an extension of vanilla graphs, where each hyperedge connects multiple nodes, thereby overcoming the limitations of pairwise modeling. As illustrated in Figure 1(b), this structure is well suited for modeling EHR, since it captures the intricate connections among medical codes and clinical visits. Accordingly, Hypergraph Neural Networks (HGNNs) [Cai et al., 2022] have been applied to process hypergraphs. However, existing HGNNs often rely on a localized two-step message propagation mechanism, thereby failing to fully utilize the patient-visit long-range dependencies within EHRderived hypergraphs. Specifically, such a defect can be described from the following two perspectives:

• Two-step (node-hyperedge-node) message propagation mechanism fails to explicitly model interactions between visits across clinically similar patients. Even when two visits involve a high overlap of shared codes,

their interactions can only be implicitly inferred via the shared nodes. Such interaction effects often necessitate multiple alternating update iterations to be fully manifested. Consequently, the long-range dependencies among patients are neglected.

• Localized propagation scheme relies solely on information from adjacent hyperedges, which overlooks long-range dependencies spanning multiple hyperedges. Hence, it limits the model's ability to capture implicit interactions among medical codes across distant visits in EHR-derived hypergraphs. For instance, a patient is initially diagnosed with obesity, but the resulting diabetes is identified only after several visits. Consequently, the long-range dependencies among visits are neglected.

In light of the aforementioned challenges, we propose PVHGL, a novel Patient-Visit-Spanned HyperGraph Learning framework for precise EHR-based diagnosis prediction. PVHGL is designed to fully utilize the patient-visit long-range dependencies within EHR-derived hypergraphs. Specifically, it first constructs a unified hypergraph containing all clinical visits of EHR data, enabling the model to capture both local and global structural properties of the hypergraph. Additionally, the framework integrates a Transformer architecture augmented with two structure-encoding matrices, enabling one-step message propagation that effectively preserves both local and global structural properties of the hypergraph. Furthermore, a medical code co-occurrence matrix is incorporated to explicitly guide the learning process by accentuating the critical interactions among medical codes. The main contributions are summarized as follows:

• Transformer-based one-step message propagation. A Transformer architecture enhanced by two structureencoding matrices for medical codes and clinical visits is designed, which preserves the local and global structural properties via efficient one-step message propagation. It overcomes the localized two-step propagation limitation of conventional HGNNs, enabling modeling of patientvisit long-range dependencies directly.
• Co-occurrence-guided attention recalibration. A medical code co-occurrence matrix is incorporated to explicitly recalibrate attention weights, which introduces the prior clinical knowledge to emphasize highorder interactions that reflect clinically meaningful dependencies.
• Extensive experiments on clinical EHR datasets. Extensive experiments are conducted on three real-world datasets, and the results demonstrate that PVHGL consistently outperforms state-of-the-art baselines in EHRbased diagnosis prediction 1 .

2 RelatedWork

2.1 Sequence-based Diagnosis Predictor

Early studies primarily utilize sequence models to process EHR for diagnosis prediction. RETAIN [Choi et al., 2016]

1 Code and Supplementary Material are available at https:// github.com/Forwhyy/PVHGL

employs two Recurrent Neural Networks (RNNs) to perform interpretable prediction. Dipole [Ma et al., 2017] incorporates a bi-directional RNN with multiple attention mechanisms to capture past and future visit information. ConCare [Ma et al., 2020] introduces a personalized clinical feature embedding framework that leverages multi-head selfattention to capture dynamic and static healthcare contexts. HiTANet [Luo et al., 2020] utilizes a time-aware encoder to learn visit-level temporal information. However, these approaches are limited in modeling the intricate dependencies among diverse medical codes [Wu et al., 2021].

2.2 GNN-based Diagnosis Predictor

GNNs have demonstrated their effectiveness in modeling EHR. For example, GRAM [Choi et al., 2017] and G-BERT [Shang et al., 2019] construct medical ontology graphs based on domain knowledge to enhance representations of medical codes. InceptionGCN [Kazi et al., 2019] improves diagnosis prediction accuracy by incorporating geometric inception modules that effectively capture structural variations in convolution operations. To uncover hidden disease relationships, Chet [Lu et al., 2022] explores horizontal connections within the medical ontology graph. DDHGNN [Wen et al., 2022] constructs a bipartite graph composed of patients and diseases, and employs a hierarchical aggregation method that accounts for the graph's spatial-temporal properties. However, these methods inherently impede a thorough exploration of high-order interactions since only pairwise relationships between medical codes are utilized [Wu et al., 2023].

2.3 HGNN-based Diagnosis Predictor

Recently, several studies have incorporated HGNNs for diagnosis prediction owing to their ability to capture high-order interactions through hyperedges. For instance, EHR2HG [Sun et al., 2022] constructs two types of hypergraphs to jointly learn patient and disease embeddings, which are aggregated to generate diagnosis predictions. HCL [Cai et al., 2022] constructs a hypergraph alongside two vanilla graphs and then applies contrastive learning to generate patient embeddings. MCHN [Zhang et al., 2024a] constructs multiple hypergraphs from different medical code views, and designs code-level and visit-level attention mechanisms to aggregate clinical information. TACCO [Zhang et al., 2024b] leverages hypergraph modeling to uncover meaningful groupings of medical concepts and visits by jointly analyzing their interactions. However, they fail to fully harness the patient-visit long-range dependencies in EHR-based hypergraphs.

3 Preliminaries

3.1 Electronic Health Records

EHR consists of longitudinal visit records collected from a cohort of patients. Formally, let R = {ru | u ∈ U} denote the complete set of visit records, where U denotes the set of patients and |U| represents the total number of individuals. Each record ru = (vu 1 , vu 2 , . . . , vu T-1 ) comprises a sequence of clinical visits, with (T -1) representing the number of observed clinical visits. Here, vij denotes the j-th visit of the i-th

(a) Unified Patient-visit Hypergraph Construction

patient. Each clinical visit v is characterized by a set of medical codes, including diagnoses, procedures, and medications, which are derived from standardized medical classification systems. To maintain clarity, we hereafter refer to clinical visit(s) as 'visit(s)' and medical code(s) as 'code(s).'

3.2 Task Definition

Let q denote the number of distinct diagnoses. Each visit at time step t is represented by a binary vector yt ∈ {0, 1}q , where yt,i = 1 indicates the presence of the i-th diagnosis. Given a patient with (T-1) historical visits, the diagnosis prediction aims to estimate the diagnosis label probabilities for the T-th visit, denoted as ˆ yT ∈ [0, 1]q .

3.3 Hypergraph Neural Networks

Conventional HGNNs typically adopt a two-step message propagation mechanism to learn node embeddings. The first step updates each hyperedge embedding by aggregating messages from its connected nodes:

where x(l) e and x(l-1) v denote the embeddings of hyperedge e at the l-th layer and node v at the (l-1)-th layer, respectively. V(e) represents the set of nodes connected to hyperedge e. In the second step, each node embedding is updated by aggregating messages from its associated hyperedges:

Here, E(v) denotes the set of hyperedges of node v. The functions fnode→edge and fedge→node serve as aggregation operators that enable message passing between nodes and hyperedges.

4 Methodology

This section presents the proposed PVHGL framework in detail. As illustrated in Figure 2, PVHGL consists of three key components: (1) unified patient-visit hypergraph construction, (2) Transformer-based one-step message propagation, and (3) co-occurrence-guided attention recalibration.

4.1 Unified Patient-visit Hypergraph Construction

A distinctive property of EHR is the inherent many-to-many structure. Specifically, each visit typically involves multiple codes, and each code often appears across multiple visits.

This characteristic induces complex high-order interactions that vanilla graphs fail to capture. To address it, we construct a unified patient-visit hypergraph that integrates visits from different patients, as depicted in Figure 2(a). Concretely, each unique code is modeled as a node, and each visit is represented as a hyperedge that connects all co-occurring codes. By concatenating hyperedges across the entire patient cohort, the unified hypergraph removes patient-specific boundaries and facilitates the joint modeling of high-order code interactions and population-wide healthcare patterns.

Formally, the unified hypergraph is represented by G = (V, E,H). Here, V = {c1, c2, . . . , cm} denotes the set of nodes, each representing a unique code, and m is the total number of codes. The hyperedge set E = {e1, e2, . . . , en} corresponds to visits, where n is the total number of visits. The topology of the unified hypergraph is denoted by the incidence matrix H ∈ Rm×n . Each entry hij is defined as:

By concatenating the local incidence matrix specific to each patient, we obtain the global incidence matrix of the unified patient-visit hypergraph.

4.2 Transformer-based One-stepMessage Propagation

Conventional HGNNs first aggregate node information via hyperedges and then propagate it back to nodes to obtain the node embeddings. Moreover, only adjacent hyperedges are processed during the learning process. Such a localized twostep message propagation mechanism fails to capture the implicit patient-visit long-range dependencies. Hence, we design a Transformer architecture enhanced by two structureencoding matrices to implement one-step message propagation, which effectively preserves both local and global hypergraph structural properties.

Local hypergraph structure learning. This module captures local structural context from both code and visit perspectives. First, we embed codes and visits into a ddimensional latent space, denoted as Zc ∈ Rm×d and Ze ∈ Rn×d , respectively. Note that visit embeddings are initialized by aggregating connected code embeddings to preserve codevisit interactions and mitigate random initialization noise.

As introduced above, the structure of the hypergraph is encoded by an incidence matrix H. Hence, it naturally facilitates the extraction of local structure information. Let Sc and Se be the local structure-encodings of codes and visits, respectively. These encodings are computed as follows:

where Wc ∈ Rn×d and We ∈ Rm×d are learnable weight matrices.

To mitigate the loss of crucial information during the local structure encoding process, we incorporate a gating mechanism that dynamically modulates the contributions of structural encodings. Specifically, we compute the scalar gates gci ∈ [0, 1] and gej ∈ [0, 1], which serve as attention weights to regulate the importance of structural information for each code i and visit j:

where β(e) j and β(v) i are learnable scalar weights corresponding to the j-th visit and the i-th code, respectively. The final local structure-aware embeddings are obtained as follows:

where ⊙ denotes element-wise multiplication applied rowwise. The matrices X and E represent the local structureaware embeddings for codes and visits, respectively. Intuitively, when a code exhibits strong correlation with specific visits, its embeddings are primarily shaped by structural information. Conversely, if the associations are weak, the model relies more on the raw input embedding. A similar interpretation applies to the gating on visits.

Global hypergraph structure learning. The codes often exhibit complex interactions that span multiple visits. Moreover, clinicians frequently leverage the experiences of similar visits in making clinical decisions. Inspired by these observations, we adopt a Transformer architecture to effectively model both the long-range dependencies among codes and the inter-visit relationships inherent in EHR.

To preserve the local structure of the hypergraph during representation learning, we incorporate the structure-aware embeddings obtained in Eq. (7) as inputs to the Transformer:

where Z (0) denotes the initial embedding matrix. We employ a self-attention mechanism to identify the most informative interactions between codes and visits, enabling the model to aggregate global structural information. It is formulated as:

where A(l) denotes the weight matrix computed as:

where W (l) Q , W (l) K , andW (l) V are learnable parameter matrices corresponding to queries, keys, and values in the l-th layer, respectively. The dimensionality parameter d determines the scaling factor of the dot products.

In our context, the explicit positional information is unavailable in the dataset. Hence, we omit positional encoding. Finally, the embeddings at the (l + 1)-th layer are obtained through a residual connection:

Compared with conventional HGNNs employing a localized two-step propagation scheme, our approach enables the joint learning of code and visit embeddings via a one-step propagation process, which eliminates the need for adjacent hyperedges. It directly computes pairwise attention across all nodes and extracts the local and global structural properties of the hypergraph.

4.3 Co-occurrence-guided Attention Recalibration

Recent studies have shown that a key challenge in existing Transformer-based models is over-globalization [Xing et al., 2024]. This issue arises because the attention mechanism tends to assign excessive weights to distant nodes, overlooking informative neighbors that are critical to downstream tasks. In clinical practice, it is widely acknowledged that certain codes frequently co-occur. Such high-frequency cooccurrence patterns typically indicate strong clinical associations. Motivated by this, we introduce a medical code cooccurrence matrix to guide attention toward clinically meaningful code pairs.

Specifically, the co-occurrence matrix M ∈ Rm×m is built by counting the frequency of code pairs that co-occur within the same visit. Note that M is only built on training data. For each visit in which codes i and j appear together, the entries mij and mji are incremented by 1. To prevent the self-attention of each code from being overly suppressed after normalization, the diagonal entry mii is set to the maximum value in the i-th row of M. Then M is normalized to align with the scale of attention scores:

After obtaining the code-to-code attention weights from Eq.(10), we recalibrate them by adding corresponding values from the co-occurrence matrix, which emphasizes attention on code pairs with strong clinical relevance:

where α is the control coefficient, and δ is an indicator function that equals 1 if both indices i and j are less than m, and 0 otherwise. The recalibrated attention weights are normalized using the softmax function:

Finally, the embedding matrix Z (l) is updated using the recalibrated attention weights ˜ A(l) following the update procedure described in Eq. (9).

4.4 Diagnosis Prediction

To effectively leverage historical visit information, we extract the sequence of visit embeddings [zu 1 , zu 2 , . . . , z u T-1 ] for patient ru from the final layer. Subsequently, a location-aware attention mechanism is applied to aggregate these embeddings into a patient embedding:

where Wa is a learnable context vector and au denotes the attention weights over visits. The final prediction is generated based on the aggregated patient embedding pu:

where ˆ yT represents the predicted probabilities for the q possible diagnoses. Let yT denote the true diagnosis labels. The model is trained by minimizing binary cross-entropy loss:

5 Experiments

5.1 Experimental Settings

Datasets. We evaluate the methods on three real-world datasets, including MIMIC-III [Johnson et al., 2016], MIMIC-IV [Johnson et al., 2023], and eICU [Pollard et al., 2018]. We retain only patients with at least two visits. Table 1 reports the dataset statistics, and the datasets are partitioned into training/validation/test sets with 70%-10%-20%.

Baselines. We divide baselines into three paradigms: (i) sequence-based models, i.e., RETAIN [Choi et al., 2016], Transformer [Vaswani et al., 2017], StageNet [Gao et al., 2020], and HiTANet [Luo et al., 2020]; (ii) GNN-based models, i.e., SafeDrug [Yang et al., 2021], CGL [Lu et al., 2021], Chet [Lu et al., 2022], Sherbet [Lu et al., 2023], and ADRL [Cheng et al., 2025]; and (iii) HGNN-based models, i.e., AllSet (Transformer) [Chien et al., 2022], HypEHR [Xu et al., 2023], MCHN [Xing et al., 2024], TACCO [Zhang et al., 2024b], and Shy [Yu et al., 2025].

Implementation Details. To ensure a fair comparison, we tune all competing methods via grid search over their hyperparameters and evaluate them on the same dataset splits. To account for stochasticity in training and initialization, we repeat each experiment five times with different random seeds and report the mean and standard deviation across runs.

Evaluation Metrics. Weighted F1 score (w-F1) and top-k recall (R@k) are selected as evaluation metrics. We set k ∈ {10, 20} since most visits have around 13 diagnoses.

5.2 Performance Comparison

As shown in Table 2, PVHGL consistently outperforms all baselines. For instance, on MIMIC-III, PVHGL yields a 24.93% improvement in w-F1 over the best sequence-based model, i.e., HiTANet. This significant gain is largely due to the limitations of sequence methods, which linearize visits and thus struggle to capture the structural dependencies among codes. We further observe that HGNN-based methods generally outperform graph-based counterparts, suggesting the importance of modeling high-order code dependencies. The HGNN-based models, except the proposed PVHGL, are restricted to message passing along the code→visit and visit→code directions. Consequently, message passing remains inherently two-stage, which makes it difficult to capture long-range code interactions and prevents explicit learning across similar visits. In contrast, PVHGL first employs two structure-encoding matrices to preserve the hypergraph's high-order semantics. It then couples code and visit embeddings to jointly model all possible code-visit interactions, enabling one-step message propagation. This design facilitates both long-range dependency modeling and explicit cross-visit knowledge sharing, ultimately leading to the best performance.

5.3 Ablation Study

To assess the contribution of each component in PVHGL, we construct three ablated variants. Specifically, PVHGLSexcludes structure encoding, PVHGLT- removes the Transformer, and PVHGLC- discards the co-occurrence matrix.

As shown in Figure 3, ablating any component results in performance degradation, indicating that each module contributes positively. The most significant drop is observed for PVHGLS-, highlighting the necessity of modeling highorder code interactions. The substantial decline observed in PVHGLT- verifies the effectiveness of the Transformer in capturing patient-visit long-range dependencies. Meanwhile, PVHGLC- yields a milder degradation, validating the contribution of high-frequency co-occurrence patterns in capturing clinically meaningful associations.

5.4 Hyperparameter Analysis

We investigate the sensitivity of three hyperparameters on MIMIC-IV: d, l, and α. As shown in Figure 4, increasing d and l improves performance initially, but further enlarging either leads to degradation, likely due to overfitting and overpropagation that impair generalization. Accordingly, α peaks at intermediate values by balancing co-occurrence information with structural signals. Deviating from this range either overfits to co-occurrence patterns or suppresses them, leading to poorer performance.

5.5 Visualization of AttentionWeights

To verify whether PVHGL captures meaningful interactions both among codes and across visits, we visualize the codelevel and visit-level attention heatmaps in Figure 5.

Code-level Attention Weights. We randomly sample ten codes, i.e., four diagnoses, three medications, and three procedures. Horizontal and vertical axes are annotated with the corresponding codes. As shown in Figure 5(a), hypotension (458.9) and pleural effusion (511.9) exhibit the highest attention score. Both are common complications of heart failure, suggesting that PVHGL effectively captures clinically meaningful co-occurrence patterns. In addition, we observe a high correlation between cardiogenic shock (785.51) and pleural effusion (511.9), which likely reflects a pathophysiological dependency. Specifically, cardiogenic shock results in significantly reduced cardiac output and systemic hypoperfusion. This hemodynamic compromise can lead to fluid overload, which in turn may cause pleural effusion. It demonstrates that PVHGL transcends simple co-occurrence patterns and successfully captures visit-spanned long-range dependencies.

Visit-level Attention Weights. We randomly sample five visits from four patients. As shown in Figure 5(b), vij denotes the j-th visit of the i-th patient. Notably, strong correlations are observed among v1 1 , v1 2 , and v3 1 . To better understand the underlying factors, we present detailed visit information in Table 3. Specifically, v3 1 and v1 1 share two diagnoses (410.71 and 414.01) and three medications, while v1 2 shares three medical codes with v1 1 . This demonstrates the presence of similar disease and treatment strategies both within and across patients. By tackling semantically related visits across different patients, PVHGL effectively captures patient-spanned long-range dependencies.

5.6 Interpretability of Prediction

PVHGL possesses satisfactory interpretability. We randomly sample a patient with six visits. The left side of Figure 6 presents the attention weights between each visit (y-axis)

and the predicted diagnoses (x-axis). Brighter regions in the heatmap indicate higher attention weights, implying that specific visits play a more influential role in predicting certain diagnoses. Figure 6(b) shows the normalized contribution of each visit to the final prediction. Notably, the fourth visit yields the highest contribution since it consistently receives high attention scores across multiple diagnosis codes. This observation suggests that PVHGL can effectively identify and prioritize clinically informative visits within a patient's longitudinal history. Such interpretability provides valuable insights into the decision-making process and aligns with clinical intuition. Hence, it enables clinicians to make more evidence-based and personalized clinical decisions promptly, such as optimizing treatment plans, identifying high-risk subgroups, and prioritizing intervention strategies.

6 Conclusion

In this work, we propose PVHGL, a novel hypergraph learning framework that addresses the limitation of capturing patient-visit long-range dependencies for EHR-based diagnosis prediction. To achieve this, we first construct a unified patient-visit hypergraph that integrates population-wide visits to model shared healthcare patterns. Further, we propose a structure-enhanced Transformer equipped with dual encoding matrices to facilitate one-step message propagation while preserving local and global structural properties efficiently. Finally, we introduce the co-occurrence-guided attention recalibration mechanism to incorporate clinical priors, emphasizing medically significant interactions. Extensive experimental evaluations demonstrate the effectiveness of PVHGL. In the future, we plan to utilize sparse attention and gradient checkpointing to reduce the quadratic computational complexity of Transformer, thereby improving the scalability of PVHGL. Moreover, we plan to collaborate with clinical doctors to validate the effectiveness of PVHGL.

Acknowledgments

This research is supported in part by the New Chongqing Youth Innovation Talent Projectand under Grant CSTB2025YITP-QCRCX0054, and in part bythe National Natural Science Foundation of China under grant 62372385.