UniSAGE: Unifying Static and Dynamic Attributes with Hyper-Structure

1 Introduction

In the era of big data, modern information systems generate and collect data that are increasingly diverse, heterogeneous, and large-scale. From financial transactions and e-commerce activities to healthcare records and social behaviors, such data are used to model a wide range of real-world entities. As data sources continue to grow in both complexity and volume, efficiently extracting meaningful representations to support downstream tasks remains a fundamental challenge [Wang et al., 2018; Xiao et al., 2022].

→Corresponding author.

This paper focuses on feature extraction from a class of complex data that we refer to as hierarchical data with both static and dynamic components. Such data typically comprise static attributes (e.g., age, gender, product category) together with dynamic records (e.g., user actions, transaction sequences, event logs). In practice, these heterogeneous attributes are often organized in semi-structured formats such as JSON, where entities exhibit hierarchical schemas with nested attributes and timestamped logs. This data paradigm is ubiquitous across real-world applications, including recommendation systems [Middleton et al., 2004; Hossain et al., 2024], healthcare records [Davis Giardina et al., 2014; Agniel et al., 2018], and financial transaction monitoring [Jayathilake, 2012; Sheble et al., 2009]. Effectively modeling such data is critical for downstream tasks such as classification [Oza and Tumer, 2008], risk assessment [Gritzalis et al., 2018], and recommendation [Javed et al., 2021].

However, effectively modeling hierarchical data with static and dynamic attributes remains challenging. First, static and dynamic attributes are often modeled separately, leading to incompatible representation spaces and fragmented semantic reasoning. As a result, capturing a holistic entity representation typically requires extensive manual model engineering, which is costly and difficult to scale. Second, many existing approaches are tightly coupled to fixed data schemas, limiting their adaptability. In real-world industrial systems, data schemas frequently evolve as attributes are added or modified, yet most models require retraining from scratch to accommodate such changes. Third, capturing interactions between static and dynamic attributes is inherently difficult. In many applications, these two attribute types are strongly interdependent. For example, in financial lending, the combination of recent loan activities (dynamic) and age-related information (static) often jointly determines default risk. Nevertheless, most existing models process static and dynamic attributes in isolation, missing crucial cross-type correlations that are essential for accurate decision-making.

To address these challenges, we propose Unifying Static and dynamic Attributes with Graph Enhancement (UniSAGE), a unified framework that models static and dynamic attributes within a shared representation space while explicitly capturing their interactions. UniSAGE first constructs a global attribute graph that reflects the hierarchical structure of the original data. Static attributes and timestamped dynamic records are represented uniformly as nodes, with static links encoding hierarchical relations and dynamic links modeling temporal dependencies. This unified graph provides a structured modeling space in which information can be propagated coherently across attribute types. The representation of an entity is obtained by aggregating information from all nodes into a root node. To ensure representational consistency, UniSAGE introduces a joint parameterization scheme with two orthogonal subspaces for static aggregation and dynamic reasoning. Orthogonality prevents mutual interference between the two modeling paradigms, allowing static and dynamic attributes to be processed independently while remaining embedded in a unified semantic space. As a result, UniSAGE no longer distinguishes static and dynamic attributes at the representation level, but only through their respective propagation paths. This design enables automatic handling of heterogeneous attributes and provides strong generalizability to evolving data schemas. Building on these unified representations, UniSAGE further constructs task-specific hyperstructures to capture higher-order interactions between selected static and dynamic attributes. These hyper-structures introduce specialized reasoning paths tailored to downstream objectives. To avoid the high computational cost of explicit hyper-structure construction, we propose Selective Semantic Aggregation (SSAgg), a lightweight mechanism that selectively aggregates task-relevant information. We theoretically show that SSAgg can simulate arbitrary hyper-structures, and empirically demonstrate its effectiveness across diverse tasks. Overall, our contributions are summarized as follows:

• We propose UniSAGE, a unified framework that models static and dynamic attributes in a shared representation space, enabling automated and generalizable learning over hierarchical data.
• We introduce SSAgg, a theoretically grounded and computationally efficient mechanism for capturing taskspecific interactions across static and dynamic attributes.
• Extensive experiments on public benchmarks and a realworld financial dataset demonstrate that UniSAGE consistently outperforms existing methods, achieving improvements of over 10% on several tasks.

2 RelatedWork

Graph-Based Encoding. Transforming complex or semistructured data into graph representations is a common paradigm in data mining and representation learning [Jeon et al., 2013; Patil et al., 2018]. Such graph-based encodings have been widely applied in recommendation systems [Wang et al., 2021], healthcare [Hassanain et al., 2022; Arch-Int et al., 2017], and finance [Zehra et al., 2021; Zhang et al., 2023; Onnela et al., 2003], where rich dependencies among heterogeneous components must be modeled explicitly. Graph neural networks (GNNs) are a natural choice for learning over such graph-structured data [Wu et al., 2020; Zhou et al., 2020]. Static GNN models [Kipf, 2016; Hamilton et al., 2017; Veliˇckovi´c et al., 2017] focus on learning representations from fixed graph structures and are effective in capturing hierarchical or relational patterns. However, they are not designed to handle temporal evolution. To address this limitation, dynamic GNNs [Skarding et al., 2021; Feng et al., 2024; Yang et al., 2024] extend static models by incorporating temporal information and evolving graph structures. Despite their success, most dynamic GNNs assume that temporal information is aligned across nodes or edges, or that dynamic interactions are defined over a shared timeline. This assumption is often violated in real-world scenarios involving heterogeneous attributes and asynchronous records, where different attributes may have varying numbers of events and non-aligned timestamps. Moreover, existing graph-based methods typically treat static and dynamic information using different modeling pipelines, which complicates representation alignment and parameter sharing. In contrast, UniSAGE adopts an attribute-level graph formulation in which both static attributes and dynamic records are treated as nodes. By jointly encoding hierarchical relations among attributes and temporal dependencies among records, this formulation avoids the need for strict temporal alignment and enables unified processing of heterogeneous information. In our experiments, we include representative static and dynamic GNN models as baselines to evaluate the effectiveness of this unified design.

Relational Deep Learning. Relational deep learning (RDL) [Fey et al., 2023; Dwivedi et al., 2025; Robinson et al., 2024; Chen et al., 2025] studies learning from relational data stored in tabular databases, where entities are described by both static attributes and dynamic records. RDL methods typically construct graphs by treating each table row (i.e., record) as a node and connecting nodes according to foreign-key relationships. This formulation has shown strong performance on benchmark datasets such as RelBench [Fey et al., 2023]. While RDL and UniSAGE address similar data modalities, their modeling strategies differ fundamentally. RDL operates at the record level and relies on schemadefined relations between rows, which limits its ability to explicitly capture hierarchical dependencies among attributes. In contrast, UniSAGE adopts an attribute-centric modeling paradigm, where both static and dynamic attributes are explicitly represented and organized according to hierarchical and temporal relations. This design enables finer-grained reasoning over heterogeneous attributes and naturally supports recursive, multi-level representations. We evaluate UniSAGE on the RelBench benchmark and directly compare it with representative RDL methods [Fey et al., 2023]. The results demonstrate that UniSAGE is more effective in modeling complex hierarchical and dynamic structures, especially in settings where attributes exhibit heterogeneous schemas and asynchronous records.

3 Preliminary

Categories of Data Attributes. In real-world applications, complex data used for modeling is commonly represented in tabular or semi-structured formats such as JSON. Attributes in such data can be broadly categorized as either static or dynamic, depending on whether their values evolve over time.

Static Attribute. A static attribute refers to a property of an entity whose value remains constant or changes negligibly over a sufficiently long period. For instance, in a credit loan system, user characteristics such as gender or place of birth are typically treated as static attributes. The value of a static attribute s, denoted as xs, depends only on the associated entity e and is defined as

where fs(·) is the mapping function corresponding to the static attribute s.

Dynamic Attribute. Dynamic attributes describe properties whose values change over time or accumulate records as time progresses. For example, a user's residential address may change over time, and a user's borrowing history grows as new loan records are added. The value of a dynamic attribute d at time t, denoted as xtd , depends on both the entity e and the timestamp t:

We represent a dynamic attribute as a time-indexed collection of records,

where fd(·) is the mapping function associated with attribute d, and T denotes the number of observed records. In practice, dynamic attributes may vary across entities in both sequence length and timestamp distribution. For example, different users typically have different numbers of loan records, and their timestamps are rarely aligned.

Overall Objective. Given an entity e with its associated static and dynamic attributes, our goal is to learn a representation he through a function ω:

where S and D denote the sets of static and dynamic attributes, respectively. The learned representation he serves as the input to downstream tasks such as classification, regression, or user modeling.

4 Methodology

4.1 Hierarchical Graph Construction

We represent each entity instance as a directed hierarchical graph G that unifies both static attributes and dynamic records. The graph is organized as a rooted tree, where edges point from sub-attributes (or earlier records) to their parents (or later records), and the root aggregates all information into an entity embedding.

Node Generation. Each attribute is instantiated as a node following the hierarchical schema. Given an attribute a, we recursively create a node va and expand its sub-attributes Sub(a) until reaching leaf attributes:

(5)

For a dynamic attribute d with T timestamped records, we treat each record as an individual node ¶vdt ♦Tt=1 (and recursively expand its sub-attributes if present):

Static Links. Static links encode the schema hierarchy: for each parent attribute aj and its (static) sub-attribute ai → Sub(aj), we add a directed edge vai ↓ vaj . We define the static adjacency matrix As → RN↑N as

When a sub-attribute is dynamic (i.e., it corresponds to a record sequence ¶dt♦Tt=1 ), connecting the parent to all records would introduce redundant paths and inflate computation. Instead, we connect the parent only to the most recent record node dT , which preserves the latest state while keeping the hierarchy concise:

Dynamic Links. Dynamic links encode temporal order within each dynamic attribute. For consecutive records of the same attribute, we add directed edges from earlier to later records:

Finally, we add a root node r and connect each top-level attribute node to r with a directed edge, forming a rooted tree. The entity representation is obtained from the final embedding of the root node r.

4.2 Orthogonal Transformation Design

UniSAGE performs bottom-up propagation on G to compute the root representation. A key challenge is that static aggregation (hierarchical composition) and dynamic reasoning (temporal modeling) typically rely on different operators (e.g., GNN/MLP vs. RNN/GRU), which can distort semantics and hinder parameter sharing even when inputs are encoded in the same embedding space. To address this, we introduce two orthogonal parameter subspaces: Ws for static aggregation and Wd for dynamic reasoning. Orthogonality encourages the two pathways to be non-interfering while keeping representations in a shared semantic space. Formally, we first map node features into a high-dimensional space H → RN↑k . We then constrain parameters for static and dynamic transformations to lie in the spans of Ws and Wd, respectively, and enforce

Here span(W) denotes the subspace spanned by columns of W. This constraint reduces cross-talk between static and dynamic transformations, stabilizing training and enabling consistent semantics under different propagation paths.

4.3 Hyper-Structure Generation

The global graph G captures hierarchical and temporal relations but does not explicitly model task-specific interactions between static and dynamic attributes. However, many decisions depend on cross-type combinations (e.g., recent behaviors conditioned on demographics). To capture such interactions, we define a task-specific hyper-structure as a subgraph G→ = (V → ↪ E → ) induced from G = (V↪ E), where V → ↗ V contains task-relevant nodes and E → connects them to form specialized reasoning paths. We then compute a hyper-structure representation via a graph encoder f :

In practice, we instantiate f with a GNN due to its strong relational reasoning ability [Xu et al., 2021; Xu et al., 2020b; Yang et al., 2022; Yang et al., 2023; Dudzik and Velickovic, 2022]. Finally, hG→ is combined with the root representation hr for downstream prediction.

4.4 Practical Implementation

We now describe the end-to-end implementation (Figure 1), corresponding to Sections 4.1-4.3.

Step 1: Global graph and node features. We construct G using As and Ad as in Section 4.1. Each node is associated with a text input that describes its attribute/value. Specifically, for leaf nodes we use "attribute name: value"; for intermediate attribute nodes we use "attribute name: [attribute flag]"; and for the root node we use a special marker "[root flag]". We encode each node text with a pretrained text encoder:

yielding the node feature matrix X → RN↑k .

Step 2: Unified propagation with orthogonal subspaces. We first project X into H:H = XW↪ (14)

where W → Rk↑k→ is learnable. We then define two subspace bases Ws → Rk→↑ks and Wd → Rk→↑kd and enforce orthogonality:

We implement this constraint by adding the orthogonality loss

while noting that numerical orthogonalization (e.g., GramSchmidt) is also applicable [Bj¨orck, 1994; Leon et al., 2013; Bj¨orck, 1967].

For static links As, we perform hierarchical aggregation where messages are mainly transformed in the static subspace while retaining complementary information from the dynamic subspace:

where h is the current representation, h↔ is the updated representation (with h↔ = h for leaf nodes), n is the number of children of vj , and ϑ↪ ¶ϖi♦ are learnable aggregation weights (analogous to attention weights in GAT [Veliˇckovi´c et al., 2017]). W+ denotes the pseudo-inverse of W.

For dynamic links Ad, we use a temporal encoder (GRU by default) to propagate along time:

where state(i) is the hidden state carried from the previous record. Eqs. 17-18 implement non-interfering static/dynamic processing: static aggregation primarily operates inWs and dynamic reasoning in Wd, while both preserve complementary information to maintain a shared semantic space. We apply these updates bottom-up from leaves to the root and obtain h↔r as the basic entity representation:

Step 3: Lightweight hyper-structure via SSAgg. Explicitly selecting V → and constructing E → can be expensive. We therefore propose Selective Semantic Aggregation (SSAgg), which implicitly simulates task-specific hyper-structures by re-propagating messages over a unified adjacency matrix:

SSAgg computes attention scores and selectively aggregates incoming messages:

where ϱ is a learnable scoring function, W→ is learnable, and ς controls selective suppression. Compared with standard softmax attention, exp(ς) in the denominator provides a tunable "gate" that can down-weight all incoming messages, enabling SSAgg to mimic node/edge selection (formalized in Section 4.5). We denote the resulting root representation as

Training objective. We concatenate the basic and enhanced representations for prediction:

and optimize the downstream loss with orthogonality regularization:

where φ balances the orthogonality constraint.

4.5 Theoretical Guarantee

In this section, we provide theoretical justification for the effectiveness of SSAgg. Recall that explicit hyper-structure construction typically involves two expensive steps: selecting task-relevant nodes and generating new edges among them. SSAgg circumvents both steps by selectively suppressing or propagating messages over the original graph. We formalize this intuition through the following propositions.

Proposition 1. Given a central node vc, a set of neighbor nodes ¶vi♦ni=1 with directed edges pointing to vc, and any fixed value of ς, there exists a scoring function ϱ of the form defined in Formula 21 such that

where ϖk↪c denotes the aggregation coefficient of node vk computed according to Formula 22.

The proof is provided in Appendix A.1. Proposition 1 shows that, by introducing the hyperparameter ς, SSAgg can arbitrarily suppress the contribution of any selected neighbor node. In other words, SSAgg can effectively block message propagation from specific nodes. This selective suppression is functionally equivalent to node selection in explicit hyperstructure construction.

We next extend this result to show that SSAgg is expressive enough to match the root-level representations produced by a broad class of hyper-structure GNNs. Specifically, we demonstrate that SSAgg can reproduce the representation obtained by applying any graph neural network to any given hyper-structure graph.

Proposition 2. Given the original graph G = (V↪ E) and an arbitrary hyper-structure graph G→ = (V → ↪ E → ) with V → ↗ V, for any graph neural network f used to encode G→ , there exists a set of SSAgg parameters ε such that

where SSAgg(·) denotes the process of generating the enhanced root representation h→r in Step 3.

The detailed proof is provided in Appendix A.2. Together, Propositions 1 and 2 establish that SSAgg can implicitly realize both node selection and structure construction. Consequently, SSAgg is theoretically guaranteed to match the representational power of explicitly constructed hyperstructures, while operating directly on the original graph with substantially lower computational overhead.

5 Experiment

In this section, we systematically evaluate the effectiveness of UniSAGE across multiple tasks. Experiments are conducted on both public benchmarks and a real-world industrial dataset. We also perform comprehensive ablation studies to analyze the contribution of each component in UniSAGE.

5.1 Overall Setup

All raw data are stored in JSON format. We compare UniSAGE with four categories of baseline methods:

• Traditional Feature ExtractionModels: we include traditional models MLP [Hornik, 1991] and LightGBM [Ke et al., 2017] as baselines. These models directly filter and extract features from the input to meet the requirements of downstream tasks.
• Static Graph Neural Networks: we also compare UniSAGE with widely-used static GNN models including GCN [Kipf, 2016], GraphSAGE [Hamilton et al., 2017] and GAT [Veliˇckovi´c et al., 2017]. These methods operate on a fixed graph structure.
• Dynamic Graph Neural Networks: to evaluate performance in evolving scenarios, we include dynamic GNNs such as TGN [Rossi et al., 2020], TGAT [Xu et al., 2020a], DyGFormer [Yu et al., 2023] and FreeDyG [Tian et al., 2024]. These models capture temporal dependencies and structural changes over time.
• Relational Deep Learning: we also consider RDL [Fey et al., 2023] and RelGNN [Chen et al., 2025] as baselines, which are designed to model complex structured relationships in data.

Detailed descriptions of all baselines are provided in Appendix B.2. For traditional feature extraction models, we use a language model to encode JSON text into representations, followed by task-specific predictors. For static GNN baselines, we construct a graph following Section 4.4 (Step 1), obtain node features via a language model, and apply GNNs on the combined adjacency matrix A = As + Ad. Dynamic GNNs follow a similar construction, treating dynamic attributes as time-evolving nodes. For RDL-based models, we strictly follow their original graph construction and learning procedures. For UniSAGE, we adopt GRU [Chung et al., 2014] as the temporal encoder in Formula 18. Hyperparameters and implementation details are reported in Appendix B.1.

5.2 Results on RelBench

Dataset Introduction. RelBench [Robinson et al., 2024] is a public benchmark for learning on relational databases, covering domains such as e-commerce, social networks, and healthcare, with dataset sizes ranging from 74K to 41M entities. We convert all tabular data into JSON format; details are provided in Appendix B.3. Due to the large scale of certain datasets, some baselines cannot be evaluated on full data. Following prior practice, we uniformly resample 2,000/500/500 instances for training/validation/testing across all tasks. Fulldataset results of UniSAGE are reported in Appendix B.6.

Entity Classification Results. Table 1 reports the AUCROC results on all entity classification tasks. We summarize the key observations as follows.

1. UniSAGE consistently outperforms all baselines. UniSAGE achieves the best results on all evaluated tasks. On average, it improves over traditional feature-based models by 15.93% and over GNN-based methods by 9.22%. In several tasks, UniSAGE exceeds the second-best method by nearly 10%, demonstrating strong and stable performance across diverse datasets.
2. UniSAGE robustly integrates structural and temporal information. While static GNNs outperform traditional baselines by 6.71% on average, their performance varies significantly across tasks. Dynamic GNNs introduce temporal signals but yield only marginal gains over static counterparts (0.52% on average), and even underperform in some cases. In contrast, UniSAGE outperforms dynamic GNNs by 8.95% on average, indicating its superior ability to jointly model heterogeneous structures and asynchronous temporal behaviors.
3. UniSAGE generalizes beyond schema-specific relational models. Compared with RDL and RelGNN, which are tailored to RelBench schemas, UniSAGE achieves an average improvement of nearly 5%. For certain tasks, such as rel-f1 driver-dnf, the gain exceeds 10%, highlighting UniSAGE's strong generalization capability on complex relational data.

Entity Regression Results. Results on entity regression tasks are reported in Appendix B.5. UniSAGE consistently achieves the best performance across all datasets, outperforming traditional models, static GNNs, dynamic GNNs, and RDL methods by 22.34%, 14.05%, 10.87%, and 8.28% on average, respectively. These results further confirm the robustness of UniSAGE across different learning objectives.

5.3 Results on Real-World Data

Dataset Introduction. We use a large-scale real-world financial behavior dataset, referred to as UserBehavior, which contains 191,927 user records with both demographic attributes (e.g., gender, education, employment status) and temporal behavioral logs (e.g., loan applications, credit account transactions). The dataset is stored in JSON format, where each user profile consists of a set of static features along with a sequence of dynamic events. The downstream task is formulated as a binary anomaly-detection problem, aiming to predict whether a user poses a potential credit risk. We use 115,156 records (60%) for training, 30,097 records (15%) for validation, and 46,674 records (25%) for testing. All baselines follow the same preprocessing protocol as in RelBench, and we use the same text encoder [Reimers and Gurevych, 2019] for feature construction. To ensure data privacy and regulatory compliance, the dataset is rigorously anonymized and all sensitive information is removed. Although the dataset is not publicly available at present, a public release is under consideration. Detailed dataset statistics are provided in Appendix B.7.

Experimental Results. Table 2 reports the performance of all baselines on UserBehavior. The results indicate that this task is challenging: simple text encoding followed by an MLP fails to produce a competitive model. Incorporating graph structure yields measurable gains, but the improvements remain modest, suggesting that existing static GNNs struggle to fully exploit the underlying hierarchical organization. Dynamic GNNs that model temporal semantics provide additional benefits, yet the overall performance is still limited, indicating that effectively leveraging asynchronous behavioral logs remains difficult. In contrast, UniSAGE achieves a substantial performance boost. Compared to the best competing baseline (DyGFormer), UniSAGE delivers a relative improvement of nearly 10%. These results underscore the advantage of UniSAGE's unified modeling of static and dynamic attributes, demonstrating its suitability for complex real-world industrial data and its ability to extract taskrelevant signals in challenging scenarios.

5.4 Ablation Study

We conduct ablation studies to quantify the contribution of each component in UniSAGE. Specifically, we evaluate on the rel-f1 driver-dnf task and the UserBehavior dataset, and compare the following variants:

1. UniSAGE without unified representation and without SSAgg (UniSAGE w/o UR and w/o SSAgg): This variant uses only Step 1 in Section 4.4 to construct the graph and perform bottom-up propagation from leaf nodes to the root. It removes unified representation learning in Step 2 and does not apply SSAgg for representation enhancement.
2. UniSAGE without orthogonality loss and without SSAgg (UniSAGE w/o OL and w/o SSAgg): Based on Variant 1, this model includes the parameter subspacesWs andWd in Step 2 but removes the orthogonality loss in Formula 16, i.e., no constraint is imposed to enforce orthogonality between the two spaces. SSAgg is also excluded.

1. UniSAGE without SSAgg (UniSAGE w/o SSAgg): This variant includes both Step 1 and Step 2 but removes SSAgg, i.e., it does not perform representation enhancement via hyper-structure reasoning.
2. UniSAGE : The full model that includes all components described in our framework.

Figure 2 presents the ablation results. Using only the graph structure yields clearly suboptimal performance (UniSAGE w/o UR and w/o SSAgg), indicating that naive propagation is insufficient. Introducing separate static aggregation and dynamic inference improves performance (UniSAGE w/o OL and w/o SSAgg), highlighting the importance of modeling both attribute types. However, without the orthogonality loss, the static and dynamic representation spaces may drift during training, which limits the effectiveness of this unified modeling. Adding the orthogonality constraint stabilizes the shared semantic space and further improves performance (UniSAGE w/o SSAgg). Finally, incorporating SSAgg enables taskrelevant hyper-structure reasoning and strengthens cross-type interactions, leading to the best overall results (UniSAGE). Overall, these results confirm the necessity of each component and align well with our theoretical motivation and empirical design choices.

5.5 Additional Results and Analysis

To improve readability and focus in the main paper, we defer several complementary experimental results and analyses to the appendix. Specifically, Appendix B.4 provides detailed statistics of all datasets and tasks used in RelBench. Complete results on the full (non-resampled) RelBench datasets are reported in Appendix B.6. Appendix B.8 presents a detailed empirical analysis of the runtime efficiency of UniSAGE, including comparisons across different datasets and task settings. Appendix B.9 provides an in-depth hyperparameter sensitivity study, analyzing the impact of key hyperparameters and demonstrating the robustness of UniSAGE under reasonable configurations.

6 Conclusion

In this paper, we propose UniSAGE, a unified framework for modeling complex data with both static and dynamic attributes. UniSAGE addresses key limitations of existing approaches by constructing a global attribute graph, learning unified representations through orthogonal parameter subspaces, and leveraging hyper-structures to enhance downstream tasks. Extensive experiments on public benchmarks and a real-world industrial dataset demonstrate that UniSAGE consistently outperforms strong baselines, achieving improvements of up to 10% on challenging tasks. By combining automation, strong generalizability, and effective modeling of implicit static-dynamic correlations, UniSAGE provides a practical and robust solution for real-world dataintensive applications.

Acknowledgments

This work was supported by National Natural Science Foundation of China (No. 62322606, No. 62441605), and the Fundamental Research Funds for the Central Universities.