FedHGN: A Federated Framework for Heterogeneous Graph Neural Networks

1 Introduction

Graph neural networks (GNNs) [Defferrard et al., 2016; Kipf and Welling, 2017; Hamilton et al., 2017; Velickovic et al., 2018; Zhang et al., 2022a; Zhang et al., 2022c] combine graph representation learning and deep learning to handle graph-structured data more effectively than traditional methods [Perozzi et al., 2014; Tang et al., 2015; Grover and Leskovec, 2016; Zhang et al., 2022b]. GNNs have various applications, such as community detection [Chen et al., 2019; Jin et al., 2019], recommender systems [van den Berg et al., 2017; Ying et al., 2018; Yang et al., 2022; Chen et al., 2022], traffic forecasting [Li et al., 2018; Zhang et al., 2018; Yu et al., 2018], and text similarity [Ma et al., 2022]. However, many real-world graphs are heterogeneous graphs, which have multiple types of nodes and edges. For instance, a financial network may consist of customers, merchants, transactions, and various relationships among them. Therefore, researchers have developed heterogeneous GNNs (HGNNs), such as RGCN [Schlichtkrull et al., 2018], to capture the complex semantics in heterogeneous graph structures.

Similar to other deep learning models, GNN performance depends on the size of the training data. If the training data is insufficient, GNNs may overfit and fail to generalize to unseen data. This problem is more severe for HGNNs, which have much larger parameter spaces [Schlichtkrull et al., 2018]. Moreover, in reality, adequate training data is often unavailable, which hinders HGNN's performance.

One possible solution to data scarcity is to collect and integrate samples from multiple parties to create a large shared dataset for model training. However, this approach raises serious privacy concerns in many application scenarios, which prevent HGNNs from using training data in a centralized way. For instance, banks and insurance companies may want to collaborate on developing an HGNN for better fraud detection. But their data, such as customer information and transaction records, are highly sensitive. They cannot transmit these data to another place for HGNN training. Therefore, there is a trade-off between data availability and data privacy. HGNNs trained separately at each client (e.g., a financial institution in this case) would perform poorly due to limited training data, while centralized training by aggregating data from all clients is not feasible due to commercial or regulatory reasons (e.g., GDPR). This leads to a question: can we train an HGNN that leverages each client's training data without violating local data privacy?

Federated learning (FL) is a promising technique to achieve this goal. Instead of sharing private data, FL clients upload model weights to a central server which aggregates them to improve the global model. The collaboratively trained model is expected to outperform the models trained locally at each client. Previous FL frameworks mainly focused on computer vision (CV) and natural language processing (NLP) tasks [McMahan et al., 2017; Li et al., 2020; Li et al., 2022]. Federated graph learning (FGL), i.e., FL on graph-structured data, has just emerged in recent years. Some works have explored FGL in homogeneous graphs [Zhang et al., 2021; Xie et al., 2021], recommender systems [Wu et al., 2021], and knowledge graphs [Chen et al., 2021;

Peng et al., 2021]. However, FL with heterogeneous graphs and HGNNs, which have a great need for collaborative training, has received little attention.

Real-world federated heterogeneous graph learning faces two unique challenges. First, unlike homogeneous graphs, heterogeneous graphs have special metadata called graph schemas. These are meta templates that describe the node types and edge types of the graph. For example, Figure 1 shows a graph schema of a scholar network. The problem is , clients may not want to share the schema information, which is essential to define an HGNN model. The graph schema reveals high-level information about how the graph is constructed, which may be sensitive in some domains, such as pharmaceutical companies and financial institutions.

Second, even if schema sharing is allowed among clients, it is usually difficult to match node/edge types of different graphs with each other, which is crucial for HGNNs to reduce complexity and share knowledge. In a federated system, different clients may use different pipelines to construct their heterogeneous graphs. For instance, one client may label their paper nodes as paper while another may call them article. This matching process may not always have the domain knowledge and human expertise available. Moreover, when a new client joins the federated system, this schema matching process needs to be repeated, and the HGNN may need to be re-defined to include new node/edge types, both of which are undesirable for industrial FL applications.

Motivated by the lack of an FGL framework for HGNNs and the challenges above, we propose FedHGN, a Federated learning framework for HGNNs that preserves schema privacy and avoids schema matching. The key idea of FedHGN is to eliminate the direct correspondence between graph schemas and HGNN model weights. We achieve this by applying schema-weight decoupling to decompose HGNN weights into shareable schema-agnostic basis weights and confidential schema-specific coefficients. To prevent the potential discrepancy between coefficients of the same/similar schemas at different clients, FedHGN employs coefficients alignment to minimize the distance between coefficients of most similar node/edge types. Thus, FedHGN enables collaborative training for better-performing HGNNs without sacrificing schema privacy or necessitating schema matching.

In summary, we make the following main contributions:

• We propose FedHGN, a novel FGL framework that enables collaborative HGNN training among clients with private heterogeneous graphs. To our knowledge, this is the first work for general federated HGNN learning.
• We propose a schema-weight decoupling (SWD) strategy to achieve schema-agnostic knowledge sharing, which preserves schema-level privacy among clients.
• We design a schema coefficients alignment (CA) component via a novel regularization term, which stabilizes FedHGN training and improves HGNN performance.
• We conduct extensive experiments on benchmark datasets with varying numbers of clients for node classification. Results show that FedHGN consistently outperforms local training and conventional FL methods.

2 RelatedWork

In this section, we review recent studies related to this work. Section 2.1 introduces graph neural networks (GNNs) and especially heterogeneous GNNs (HGNNs). Section 2.2 introduces conventional federated learning (FL) and recent efforts on federated graph learning (FGL).

2.1 Graph Neural Networks

Graph neural networks (GNNs) are designed to overcome the limitations of shallow graph embedding methods, such as DeepWalk [Perozzi et al., 2014] and node2vec [Grover and Leskovec, 2016]. Taking advantage of the neural network architecture, GNNs can leverage node attributes and benefit from various training paradigms of deep learning. The general idea of GNNs is to characterize each node by its own features and its local neighborhood's information. Following this idea and combined with graph signal processing, spectral-based GNNs like ChebNet [Defferrard et al., 2016] and GCN [Kipf and Welling, 2017] were first developed to perform convolutions in the Fourier domain of a graph. Then, by allowing convolutions in the graph domain, spatial-based GNNs like GraphSAGE [Hamilton et al., 2017] and GAT [Velickovic et al., 2018] were proposed to improve scalability and generalization ability, with many follow-up studies [Xu et al., 2019; Wu et al., 2019].

Heterogeneous GNNs

Conventional GNNs assume homogeneous input graphs with single node/edge types. However, real-world graphs are usually heterogeneous, having multiple node/edge types. To capture complex structural and semantic heterogeneity in these graphs, researchers proposed heterogeneous GNNs (HGNNs). As an early attempt, RGCN [Schlichtkrull et al., 2018] aggregates neighborhoods using relation-specific weights. Many other HGNNs follow similar ideas to project and aggregate neighborhoods based on node/edge types, including HetGNN [Zhang et al., 2019], HGT [Hu et al., 2020], Simple-HGN [Lv et al., 2021], etc. Another line of research considers meta-paths to capture complex high-order relationships between nodes, such as HAN [Wang et al., 2019b], MAGNN [Fu et al., 2020], and GTN [Yun et al., 2019].

Although many HGNNs claim to have good performance in real-world applications, none have considered the federated setting with privacy concerns across clients. They simply assume a centralized setting with all data readily available. With privacy restrictions, HGNNs trained with limited local data could suffer from degraded performance and biased predictions. Our proposed FedHGN offers a feasible way to collaboratively train better-performing HGNNs among multiple clients without sharing local graph data and graph schemas.

2.2 Federated Learning

Federated learning (FL) is a machine learning setting for collaborative cross-client model training without sharing raw data [Zhang and Zhu, 2020; Zhang et al., 2023]. Instead of gathering data to a central server, FL keeps data localized on each client and only shares model weights or gradients. Multiple FL algorithms have been proposed for regular data in Euclidean space, such as FedAvg [McMahan et al., 2017], FedProx [Li et al., 2020], and SCAFFOLD [Karimireddy et al., 2020]. These FL algorithms are applied in CV and NLP, where data samples are naturally isolated from each other. But for graphs, especially in node-level and link-level tasks, nodes and edges as data samples are associated with each other. Naively migrating conventional FL frameworks might not bring optimal results for graph data.

Federated Graph Learning

Recently, some researchers have proposed FL methods for graphs, which are coined as federated graph learning (FGL). GCFL [Xie et al., 2021] employs a client clustering mechanism to reduce data heterogeneity in federated graph classification. FedSage [Wu et al., 2021] generates missing neighbors to mend dropped cross-client links caused by data isolation in an FGL system. FedGNN [Wu et al., 2021] conducts GNN-based recommendations on user devices with a privacypreserving FGL framework. FedE [Chen et al., 2021] and FKGE [Peng et al., 2021] are federated algorithms for knowledge graph embedding (KGE). FedAlign [Lin et al., 2020] applies basis alignment and weight constraint on federated RGCN to improve personalization and convergence.

However, few FGL algorithms have considered heterogeneous graphs and HGNNs. Aside from graph data (nodes/edges) and optional attributes, heterogeneous graphs also have special metadata called graph schemas. Naively applying existing FGL methods would lead to schema privacy breaches or require schema matching. FedE, FKGE, and FedAlign are the most related research works to our FedHGN, but they still have some notable limitations. Due to the inherent restriction of KGE algorithms, FedE and FKGE assume all or part of the entities/relations are already matched among clients. While our FedHGN does not require any entity/relation matching, reducing privacy concerns and human labor. FedAlign focuses on a particular HGNN model, namely RGCN. On the other hand, our FedHGN is a general framework compatible with almost any HGNN architecture. Moreover, FedHGN is equipped with the coefficients alignment component that can alleviate the discrepancy between same-type schema coefficients at different clients.

3 Preliminary

This section gives formal definitions of some important graph and FL terminologies related to this work. Table 1 provides a quick reference to the notations used in this work.

Definition 1 (Heterogeneous Graph). A heterogeneous graph is defined as a graph G = (V, E) associated with a node type mapping function ϕ : V → A and an edge type mapping function ψ : E → R. A and R denote the pre-defined sets of node types and edge types, respectively, with |A| + |R| > 2.

The graph schema of G is given by TG = (A,R), which is essentially a graph of node types connected by edge types.

Definition 2 (Heterogeneous Graph Neural Network). A heterogeneous graph neural network (HGNN) is a GNN model designed for representation learning on heterogeneous graphs. Generally, an HGNN layer can be formulated as

where Aggr(·) is an edge-type-specific neighborhood aggregation function parameterized by θr, Reduce(·) is a function to combine (e.g., sum) aggregation results from different edge types, and Transϕ(v)(·) is a node-type-specific transformation parameterized by θϕ(v). Some HGNNs might contain additional schema-irrelevant parameters θc.

Definition 3 (Federated Graph Learning). A federated graph learning (FGL) framework consists of a server and K clients. The k-th client holds its own graph dataset Dk = (G k ,Xk [,Yk ]), where G k = (V k , E k ) is the graph, Xk is the node feature matrix and Yk is the optional label matrix. The goal of FGL is to collaboratively train a shared graph model without disclosing clients' local data.

Definition 4 (Federated Heterogeneous Graph Learning). Federated heterogeneous graph learning is a sub-category of FGL, where the graph G k owned by each client is heterogeneous. The graph schema TGk = (Ak , Rk ) could be different or even kept private across clients.

4 Methodology

In this section, we propose FedHGN, a novel FGL framework for heterogeneous graphs and HGNNs. FedHGN aims to achieve improved HGNN performance on each client through collaborative training under a federated framework without sharing graph data, node/edge attributes, and graph schema. To attain this goal, FedHGN needs to separate graph schema from model weights before collecting and aggregating weights on the server side. Therefore, we first propose a schema-weight decoupling (SWD) strategy to enable schema-agnostic knowledge sharing in Section 4.1. Then, to alleviate the adverse side effects introduced by the decoupling strategy, we design a coefficients alignment (CA) component in Section 4.2. Next, the overall FedHGN framework is depicted in Section 4.3. Finally, a privacy analysis of our proposed framework is conducted in Section 4.4. A graphical illustration of FedHGN is provided in Figure 2.

4.1 Schema-Weight Decoupling

If naively combining HGNN and conventional FL methods (e.g., FedAvg), the server and the clients would need to first coordinate for a global graph schema. Taking Figure 3 as an example, the resulting global graph schema is generated by matching the two clients' local schemas, which inevitably breaches schema privacy and involves human expertise.

To avoid the privacy-sensitive schema matching process, we propose SWD to remove the correspondence between graph schemas and HGNN weights. For schema-specific weights θks ∈ Rd of any node/edge type s ∈ A ∪ R from an HGNN at client k, one can apply basis decomposition by

where βk s,i ∈ R are schema-specific coefficients, vk i ∈ Rd are basis weights (or bases), and B is a hyperparameter specifying the number of bases. Here model weights are treated as flattened vectors for notational brevity. Without loss of generality, SWD can be easily extended to higher-dimensional weights. The bases vk i are shared by all node/edge types, and not bound to the graph schema. They can be trained along with schema-irrelevant weights θkc under the conventional FL paradigm. On the other hand, a small number of coefficients βks ∈ RB are schema-dependent. They are tuned locally at each client and do not participate in weight aggregation at the server. By applying SWD and sharing only bases, FedHGN achieves schema-agnostic knowledge sharing, which avoids graph schema leakage and manual schema matching.

4.2 Coefficients Alignment

Due to the randomness throughout training, the non-convex nature of the objective, and the non-IID heterogeneous graph data among clients, schema-specific coefficients βks corresponding to the same node/edge type at different clients might diverge from each other towards distinct directions in the FL training process. This discrepancy between schema coefficients among clients may lead to unstable training, prolonged convergence, and sub-optimal results. Suppose the central server knows the matchings between schema coefficients among clients, this discrepancy issue can be easily resolved by aggregating and synchronizing them between the server and the clients. However, this strong assumption violates schema privacy and requires schema matching, which is not allowed in federated heterogeneous graph learning.

To address the discrepancy issue without compromising the setting, we devise a heuristic approach to align the schema coefficients by enforcing a client-side loss constraint onto them. Intuitively, even though schema coefficients of the same node/edge type at different clients may diverge in different directions, there is a high chance of them being closer in the vector space than those of the different or dissimilar node/edge types. During each communication round, in addition to the shareable bases vk i and schema-irrelevant weights θkc , the FedHGN server also collects and distributes

Algorithm 1 FedHGN framework.

Require: K clients indexed by k; Client sampling fraction Learning rate η; Alignment regularization factor λ.

• C; Number of bases B; Number of local epochs E; 1: procedure SERVEREXECUTION 2: Initialize [θc]0 , [v]0 , and {[βk ]0 : 1 ≤ k ≤ K} (subscripts of v and β omitted for brevity) 3: for round t = 1, 2, . . . do 4: St ← (random set of max(C · K, 1) clients) 5: for each client k ∈ St in parallel do 6: [θkc ]t , [vk ]t , [βk ]t ← ClientUpdate( k, [θc]t-1 , [v]t-1 , {[βk′ ]t-1 : k′ ̸= k}) 7: end for 8: ([θc]t , [v]t ) ← ∑k∈St Nk NSt ([θkc ]t , [vk ]t ) 9: [βk ]t ← [βk ]t-1 , ∀k /∈ St 10: end for 11: end procedure 12: procedure CLIENTUPDATE(k, θc, v, {βk′ : k′ ̸= k}) 13: θs ← ∑Bi=1 βk s,ivi, ∀s ∈ Ak ∪ Rk 14: for epoch i = 1, 2, . . . , E do 15: for minibatch m sampled from Dk do 16: Lktask ← (task-specific loss given m) 17: Lkalign ← Equation (3) 18: θc, v, βk ← (θc, v, βk )-η∇ (Lktask + λLkalign) 19: end for 20: end for 21: Return θc, v, and βk to server
• 22: end procedure

the schema-specific coefficients βks from/to clients. But instead of aggregating them on the server side, FedHGN introduces a novel regularization term on the client side to penalize the local node/edge types' coefficients deviating from the most similar counterpart from other clients:

where βks ∈ RB is the schema-specific coefficients of node/edge type s ∈ Ak ∪ Rk from client k's HGNN model. The key here is how to identify the most similar counterpart for βks . We intuitively select the one with the smallest distance by taking the minimum, which has the highest chance of matching the node/edge type. By minimizing this heuristic regularization term, FedHGN aligns schema coefficients across clients without disclosing actual graph schema.

4.3 Framework Overview

Our FedHGN framework largely follows FedAvg, which iteratively conducts local training at multiple clients and weight aggregation at a central server. In addition to the typical operations of FedAvg, FedHGN employs SWD and CA to preserve schema privacy without performance degradation. Before starting the federated training process, the server and the clients would negotiate for the hyperparameters of the HGNN model used, such as the number of bases (B) in SWD. The primary training process comprises many communication rounds between the server and the clients. The detailed server-side and client-side operations of a single round in FedHGN are described below. Figure 2 illustrates this training process. Complete pseudocode is given in Algorithm 1.

Server Operations. At round t, the server first samples a fraction of clients St for this round of training. The server then sends the latest model parameters to each client k ∈ St, including the aggregated schema-agnostic weights, [θc]t-1 and [vi]t-1 , and collected schema coefficients [βk′ s ]t-1 from the previous round. After client-side local updates, the server receives the updated [θkc ]t , [vk i ]t , and [βks ]t from each k ∈ St. Finally, the server aggregates the schema-agnostic weights by [θc]t = ∑k∈St Nk NSt [θkc ]t and [vi]t = ∑k∈St Nk NSt [vk i ]t , where Nk is the number of training samples at client k, and NSt = ∑k∈St Nk is the total number of training samples of this round. The server also sets [βks ]t = [βks ]t-1 for k /∈ St.

Client Operations. At round t, each selected client k receives the latest model parameters from the server, including schema-agnostic weights, [θc]t-1 and [vi]t-1 , and other clients' schema coefficients [βk′ s ]t-1 . The schema-specific weights are recreated via Equation (2). Then, the client would conduct local training using local heterogeneous graph data Dk for several epochs to tune model parameters. The clientside objective function is a sum of the task-specific loss and the CA regularization term: Lk = Lktask + λLkalign , where Lktask could be cross entropy for node classification tasks or some ranking loss for link prediction tasks. Finally, the client uploads the updated schema-agnostic HGNN weights, [θkc ]t and [vk i ]t , and schema coefficients [βks ]t back to the server.

4.4 Privacy Analysis

In this section, we analyze the communications between the server and the clients and discuss whether FedHGN can preserve the data privacy of local graph schemas.

Server-side Analysis. FedHGN server is restricted to collecting limited information that cannot be used to infer graph schemas of any clients. The central server receives only HGNN model weights vk i , θkc , and βks from each client. vk i and θkc are schema-agnostic model weights, revealing no schema information. Although βks are schema-specific coefficients, the server can only deduce the number of node/edge types in each client. It still cannot learn other meaningful schema information, like the matching among them or the physical name of their corresponding node/edge types.

Client-side Analysis. Data privacy at each client is well protected under the FedHGN framework. Each client k receives vi, θc, and βk′ s (for k′ ̸= k) from the server. Similarly, clients cannot infer other clients' graph schemas from vi and θc. Since the schema-specific coefficients βk′ s are sent to client k as an unordered set, client k cannot differentiate the source client of each received βk′ s . Therefore, neither the HGNN model nor the graph schema of any other client can be derived based on what client k receives from the server.

5 Experiments

In this section, we demonstrate the effectiveness of FedHGN on federated learning of HGNNs by conducting experiments for node classification on a series of heterogeneous graph datasets. The experiments are designed to answer the following research questions. RQ1: Can FedHGN achieve a better HGNN performance than purely local training? RQ2: Can FedHGN outperform conventional FL methods in training HGNNs? RQ3: What are the effects of the proposed schema-weight decoupling and coefficients alignment? RQ4: How is FedHGN affected by the hyperparameters?

5.1 Experimental Setup

Datasets. We select widely-adopted heterogeneous graph datasets for our node classification experiments: AIFB, MUTAG, and BGS preprocessed by Deep Graph Library (DGL) [Wang et al., 2019a]. These datasets are highly heterogeneous with more than 50 node/edge types. To simulate the federated setting, we randomly split each dataset into K = 3, 5, and 10 clients. We propose two random splitting strategies for heterogeneous graphs: (1) Random Edges (RE) which randomly allocates edges to K clients and (2) Random Edge Types (RET) which randomly allocates edge types to K clients. RE simulates a scenario where each client owns a part of the complete graph, within which there are overlapping nodes/edges among clients. RET simulates a more complex scenario where clients construct the local heterogeneous graphs differently, resulting in distinct graph schemas. Averaged statistics of the BGS dataset split into K clients are summarized in Table 2. More details about the datasets and splitting strategies are provided in the appendix 1 .

Baselines. To assess FedHGNand demonstrate its superiority, we choose the following settings/algorithms as baselines: (1) centralized training, (2) local training, (3) FedAvg, and (4) FedProx. We do not include existing FGL methods as baselines because they cannot be easily adapted to HGNNs. FedAlign is not adopted because its source code is not available and some design details are not disclosed in its paper.

Implementation. We employ an RGCN-like architecture as the HGNN model. Other details are in the appendix.

1 Appendix can be found via https://arxiv.org/abs/2305.09729

5.2 Experimental Results

We report results averaged from 5 runs using different random seeds. The accuracy score of each run is computed via a weighted average of the K clients' local testing scores based on the numbers of their testing samples.

Main Results (RQ1 & RQ2)

To evaluate FedHGN against local training and conventional FL algorithms, we conduct comprehensive experiments on three datasets with varying client numbers. Note that schema sharing is required and allowed in FedAvg and FedProx, but not in FedHGN. Averaged accuracy scores with standard deviations are listed in Table 3. Due to the space limit, we leave the K = 3 settings of MUTAG and BGS in the appendix.

As shown in the table, FedHGN consistently outperforms local training and conventional FL algorithms in most settings. As expected, HGNN performance deteriorates in local training due to insufficient data, especially on AIFB and BGS. But conventional FL algorithms (FedAvg and FedProx) do not have a noticeable advantage over local training in our experiments. On AIFB (RE), they are even outperformed by the local setting. Although FedProx is designed to alleviate the non-IID data issue, it cannot correctly handle heterogeneous graph data based on our results. Compared to FedAvg and FedProx, FedHGN makes no compromise in schema-level privacy and yet achieves superior results, showing that FedHGN is a better option for federated HGNNs. FedHGN's advantage in preserving HGNN performance might come from the decoupled schema coefficients. They may serve as personalized model parameters to help each client combat the severe non-IID problem of heterogeneous graphs.

Ablation Study (RQ3)

To analyze the effects of the proposed designs, we compare variants of FedHGN on MUTAG (RE) and BGS (RE) in Table 4 by applying different schema-weight decoupling (SWD) strategies and whether employing coefficients alignment (CA) or not. The table shows that CA brings about a 7% improvement compared to the variant without it. We also notice a longer convergence time when removing CA. Therefore, the alignment component is necessary for FedHGN to stabilize training and maintain performance. For SWD, we observe that additionally aggregating schema coefficients at the server side (breaking schema privacy) leads to some performance gain, which can be well compensated by CA. Another interesting observation is that the FedHGN variant that only aggregates schema coefficients achieves decent results. We suspect it may resemble the local setting because the shared coefficients have a small parameter space. We also present the results produced by removing both SWD and CA, which reduces FedHGN to FedAvg. Although outperforming the variant that uses only SWD, FedAvg violates schema privacy and is still inferior to the full version of FedHGN. All these results confirm the effectiveness of SWD and CA.

Hyperparameter Analysis (RQ4)

To investigate how FedHGN is affected by the newly introduced hyperparameters, we conduct pilot tests on AIFB with different choices for the number of bases B and the alignment regularization factor λ. Except for the concerning hyperparameter, all other setups are kept unchanged. From Figure 4, we find that FedHGN is sensitive to the choice of B. The optimal performance is reached at around B = 35 and B = 20 for AIFB (RE) and AIFB (RET), respectively. Intuitively, B should be related to the number of edge types involved in the FL system. Since other settings (MUTAG and BGS) contain a similar or smaller number of edge types, we roughly choose B = 20 as a sugar point to report the main results.

For the alignment regularization factor λ, FedHGN performance does not change much when λ is reasonable, i.e., for λ ≤ 5. But when λ gets larger than 5, the testing accuracy drops dramatically on both AIFB (RE) and AIFB (RET). When λ is too large, the optimization objective is dominated by the CA regularization term Lalign. HGNNs cannot learn much knowledge from the task-specific loss Ltask in this case. Hence, we just set λ = 0.5 for our main experiments.

6 Discussion

This work aims to avoid schema matching and ensure schema privacy during federated HGNN learning. By carefully examining the HGNN architecture, we notice the crux of the matter is the correspondence between model weights and node/edge types. Our FedHGN is designed to decouple this binding and achieve schema-agnostic weight sharing. This seems unnecessary as one could just apply secure multiparty computation (MPC) techniques like private set operations [Freedman et al., 2004; Kissner and Song,

2005] to match the node/edge types across clients. However, node/edge types' naming conventions may differ across clients. Human labor is usually inevitable to coordinate a set of naming rules, which generally breaks schema privacy. Furthermore, without SWD, the server can derive client schema from received gradient/weight changes between communication rounds. Although gradients can be protected by differential privacy [Abadi et al., 2016], homomorphic encryption [Phong et al., 2018], secure aggregation [Bonawitz et al., 2017], or MPC [Mohassel and Zhang, 2017], these approaches could affect model performance, increase computational overhead, or impede communication reduction methods [Erg¨un et al., 2022]. Therefore, we reckon FedHGN as a practical design to reduce schema privacy risks.

7 Conclusion

This paper proposes FedHGN, a novel federated heterogeneous graph learning framework for HGNNs with schema privacy protection. By decoupling graph schemas and model weights, FedHGN removes the correspondence between node/edge types and HGNN weights to achieve schemaagnostic knowledge sharing. With an alignment regularization term, the discrepancy among same-type schema coefficients across clients is alleviated to stabilize training and boost model performance. Experimental results show that FedHGN can bring consistent performance improvement to HGNNs while preserving both data and schema privacy, outperforming local training and conventional FL algorithms.

Acknowledgments

The work described here was partially supported by grants from the National Key Research and Development Program of China (No. 2018AAA0100204) and from the Research Grants Council of the Hong Kong Special Administrative Region, China (RGC GRF No. 2151185, CUHK 14222922).