Learning Causal Effects on Hypergraphs (Extended Abstract)

1 Introduction

Group interactions among individuals exist in many scenarios, e.g., massive gathering events. Although the conventional pairwise graph definition covers various applications (e.g., physical contact networks or social networks [Mitchell, 1974]), it fails to capture the complete information of group interactions (where each interaction may involve more than two individuals) [Bai et al., 2021; Feng et al., 2019]. Hypergraphs can be introduced to address this limitation. Consider a hypergraph example that individuals are connected via in-person social events, each event can be represented as a hyperedge (Fig. 1a). Each hyperedge can connect any number of individuals, unlike an ordinary edge connecting exactly two nodes (Fig. 1b).

∗This is an extended abstract of a paper [Ma et al., 2022] that won the Best Paper Award at KDD 2022.

𝑢ହ

• (c) First, second, and third-order interferences with u1.

Figure 1: (a) An illustrative example of group interactions on a hypergraph, where each circle represents a hyperedge (group); (b) An ordinary graph projected from this hypergraph; (c) Illustration of interferences with u1 from its neighbors on the hypergraph. Note interference on (b) is pairwise (first-order only) while higher-order interference exists on the original hypergraph (a).

While many studies have been devoted to utilizing such a generalized hypergraph structure to facilitate machine learning tasks [Bai et al., 2021; Feng et al., 2019], the majority were still executed at the statistical correlation level. A critical limitation here is the lack of causality, which is particularly important for understanding the impact of a policy intervention (e.g., wearing face covering) on an outcome of interest (e.g., COVID-19 infection). For individuals connected as in Fig. 1a, one may ask "how would each individual's face covering practice (treatment) causally influence their infection risk (outcome)?" This causal problem is particularly hard on hypergraphs, since the outcome of each individual is not only affected by their own treatment, but also influenced by the treatment of other individuals on the hypergraph (e.g., face covering practice of other individuals who may physically contact the target individual through a gathering event).

We focus on learning causal effects on hypergraphs. Specifically, we aim to estimate the individual treatment effect (ITE) under hypergraph interference from observational data. Classic ITE estimation relies on the Stable Unit Treatment Value (SUTVA) assumption [Fisher, 1936; SplawaNeyman et al., 1990] that there is no interference [Tchetgen and VanderWeele, 2012; Hudgens and Halloran, 2008] (i.e., spillover effect) among individuals/instances (also referred to as units in causal inference literature). That means the outcomes for any instance are not affected by the treatment of other instances. This assumption is often impractical in the real world, especially on graphs where the interference is ubiquitous [Ahluwalia et al., 2001; Yilmaz et al., 2002]. Most existing studies of interference [Aronow and Samii, 2017; Basse and Feller, 2018; Imai et al., 2020; Kohavi et al., 2013; Tchetgen and VanderWeele, 2012; Ugander et al., 2013; Yuan et al., 2021; Ma and Tresp, 2021] assume the interference only exists in a pairwise way on ordinary graphs. This pairwise interference notion is insufficient to characterize the high-order interference on hypergraphs. As shown in Fig. 1c, within a gathering event (hyperedge) between u1, u2 and u3, an individual's (u1) infection outcome can be affected by the first-order interference from other individuals (u2 → u1 and u3 → u1) as well as the high-order interference from the interactions among other individuals (the interaction between u2 and u3 may also act on influencing the exposure of the virus to u1; consequently, u1's infection risk can be affected by this second-order interaction effect, i.e., u2 × u3 → u1). This demands techniques capable of modeling high-order interference, but to the best of our knowledge, very little work has been done in this area.

We propose a novel framework- Causal Inference under Spillover Effects in Hypergraphs (HyperSCI)-to model high-order interference and promote ITE estimation performance. At a high level, this framework controls for the confounders and models high-order interference based on representation learning, then estimates the outcomes based on the learned representations. We evaluate HyperSCI through extensive experiments under different scenarios of high-order interference, and provide in-depth analysis of how our framework acts on different nodes and hyperedges.

2 ProblemDefinition

Definition 1. (Hypergraph) A hypergraph H = {V, E} consists of a set of n nodes V = {vi}ni=1 and a set of m hyperedges E = {ek}mk=1 . Each hyperedge can connect any number of nodes.

In the studied problem, the given observational data is denoted by {X,H,T,Y}. Here, X = {xi}ni=1 , T = {ti}ni=1 and Y = {yi}ni=1 represent node features, treatment assignments, and observed outcomes, respectively. H = {hi,e} ∈ Rn×m is an incidence matrix for hypergraph H. Here, hi,e = 1 if node i is in hyperedge e, otherwise hi,e = 0. The treatment assignment for each node is binary (i.e., ti ∈ {0, 1}). We use H,X, T to denote the random variables for the hypergraph structure, features, and treatment for any node.

Definition 2. (Potential outcome) The potential outcome [Rubin, 1980] of the unit i (denoted by y1 i or y0 i ) is defined as the outcome which would be realized for unit i under treatment ti = 1 or ti = 0. These potential outcomes can be obtained via a transformation function Y Ti i = ΦY (Ti,Xi, T-i,X-i,H). Here, ΦY is a (non-deterministic)

function, i.e., yti i = ΦY (ti, xi,T-i,X-i,H), where (·)-i denotes all other nodes on H except i.

Our aim is to estimate ITE in a hypergraph. The ITE in the studied problem is defined as follows:

Definition 3. For each node i on the hypergraph H, the individual treatment effect (ITE) τ(xi,T-i,X-i,H) is defined by the difference between potential outcomes corresponding to ti = 1 and ti = 0:

Here, we give our main assumptions and theoretical analysis for this study:

Assumption 1. (Expressiveness of summary function) There is an expressive summary function oi = SMR(H,T-i,X-i) to summarize the environmental information for each node. For any node i, any values of H,X-i, and T-i, if the output oi is determined, then the value of the potential outcomes y1 i and y0 i with feature xi are also determined.

Assumption 2. (Unconfoundedness) For any node i, given the node features, the potential outcomes are independent with the treatment assignment and summary of neighbors, i.e., Y 1 i , Y 0 i ⊥⊥ Ti,Oi|Xi.

Theorem 1. (Identifiability) The defined ITE can be identifiable from observational data under the above assumptions.

We refer readers to [Ma et al., 2022] for a full derivation.

3 ProposedMethod

Fig. 2 shows an overview of our framework HyperSCI, which contains three main components: confounder representation learning, interference modeling, and outcome prediction.

3.1 Confounder Representation Learning

HyperSCI learns representations of confounders by mapping the node features xi into a latent space with a multilayer perceptron (MLP) module, i.e., zi = MLP(xi). The confounder representations for all the nodes are denoted by Z = {zi}ni=1 . Similar as [Shalit et al., 2017], a Wasserstein-1 distancebased representation balancing method is used to minimize the distance between the representation distributions of the treatment group and control group, aiming to improve the treatment effect estimation performance.

3.2 Interference Modeling

An interference modeling module is developed to model the high-order interference among nodes in the hypergraph. More specifically, a function Ψ(·) is learned via a hypergraph neural network module to obtain the interference representations (pi) for each node i, i.e., pi = Ψ(Z,H,T-i, ti). The illustration of this module is shown in Fig. 3. This module is implemented based on a hypergraph convolutional network [Bai et al., 2021; Yadati et al., 2019] as well as a hypergraph attention mechanism [Bai et al., 2021; Zhang et al., 2019; Ding et al., 2020].

Hypergraph module

$#

Interference

representation

Figure 3: An illustration of the hypergraph module in HyperSCI. Here node v1 (highlighted in yellow) is taken as an example.

To learn the interference representations for each node, the treatment and confounder representations are propagated through the hypergraph structure. A vanilla Laplacian matrix for the given hypergraph H can be calculated as:

where D ∈ Rn×n is a diagonal matrix in which each element stands for the node degree (i.e., ∑me=1 hi,e). B ∈ Rm×m is a diagonal matrix in which each element corresponds to the size of each hyperedge (∑ni=1 hi,e). The hypergraph convolution operation is defined as:

where P(l) denotes the representations in the l-th layer of the hypergraph module. The input of the first layer is the confounder representation masked by the treatment assignment, i.e., p(0) i = ti ∗ zi. Here, ∗ is element-wise multiplication. W(l+1) ∈ Rd(l)×d(l+1) represents the parameter matrix in the (l+1)-th layer of the hypergraph module, where d(l) and d(l+1) are the dimensionality of the l-th layer and (l+1)-th layer, respectively.

While the hypergraph convolution layer allows for interference modeling through hyperedges, it lacks flexibility to consider the varying significance of interference on different nodes via different hyperedges. To address this, a hypergraph attention mechanism [Bai et al., 2021; Ding et al., 2020; Zhang et al., 2019] is utilized to capture the intrinsic relationship between nodes and hyperedges. Specifically, the attention weights are learned for each node and its corresponding hyperedges, which allows for a better understanding of how certain individuals, such as those participating in group events, may have a greater influence on or be influenced by others in these groups within the context of a hypergraph, as where σ(·) is an activation function, Ei is the set of hyperedges which contain the node i. ze is the representation for each hyperedge e, obtained by aggregating across the representations of its associated nodes. Wa denotes a parameter matrix to compute the node-hyperedge attention. sim(·) denotes a similarity function, which can be implemented as:

Here, a is a weight vector, [·, ·] is a concatenation operation. The attention scores are used to model the different significance of interference. More specifically, the original incidence matrix H of the hypergraph in Eq. 2 is replaced with an attention-involved matrix ˜H = {˜hi,e}, where ˜hi,e = αi,ehi,e.

3.3 Outcome Prediction

Based on the confounder representations and the interference representations, the potential outcomes are predicted by:

where f1(·) and f0(·) are learnable functions which are trained to predict potential outcomes for treatment assignment 1 and 0, respectively. The ITE for each node i is then estimated by: ˆ τi = ˆ y1 i - ˆ y0 i . The prediction for the observed outcome is obtained by ˆ yi = ˆ yti i . The final loss function is:

where the first term is the outcome prediction loss, which can be implemented by standard mean squared error. Lb is the representation balancing loss, as introduced in Section 3.1. Θ denotes all the model parameters. α and λ are hyperparameters that control the weights for representation balancing and model regularization, respectively.

4 Experimental Evaluation

It is often very hard to obtain the ground-truth counterfactuals as only one of the two potential outcomes can be obtained in the observational data. Hence, we follow a standard practice to evaluate our framework and the alternative approaches on three semi-synthetic datasets Contact, GoodReads, and Microsoft Teams. We leverage as much real-world information as possible in the simulated environment. Our datasets are all based on real-world hypergraphs, and we retain the treatment allocations as well as node features (covariates) if they are available. We simulate the outcome generation process to assess the true ITEs, which allow us to evaluate the performance of ITE estimation. Full details of the datasets, baselines, and experimental settings can be found in [Ma et al., 2022].

4.1 ITE Estimation Performance

The performance of ITE estimation in hypergraph is shown in Table 1. HyperSCI outperforms all the baselines under different (linear and quadratic) simulation settings of the outcome generation function. As for the reasons, HyperSCI can leverage the hypergraph structure to model the high-order interference. In this way, it mitigates the influence of the interference on ITE estimation performance. Among baselines, some of them consider the pairwise network interference (GCN-HSIC and GNN-HSIC [Ma and Tresp, 2021]), or use the graph structure to infer the hidden confounders (Netdeconf [Guo et al., 2020]). These methods perform better than those baselines (LR, CEVAE [Louizos et al., 2017], CFR [Shalit et al., 2017]) which cannot handle graph information. Furthermore, in the simulation, the hyperparameter β controls the level of hypergraph spillover effect in the outcome simulation. The ITE estimation results under different values of β are shown in Fig. 4. When β increases, the outcome is more strongly affected by interference, and larger performance gains can be observed from HyperSCI compared with the baselines.

4.2 ACloser Look at High-Order Interference

We further take a closer look at high-order interference. We investigate how HyperSCI responds to hyperedges with different sizes. More specifically, we remove the hyperedges with size larger than k, denote the modified hypergraph as H(k) , and vary the value of k. In Fig 5, we compare the ITE estimation performance of HyperSCI with its variant on the projected ordinary graph HyperSCI-G. We observe that: 1) When k = 2 (hyperedge size ≤ 2), the performance of HyperSCI-G is close to HyperSCI. Because when k = 2, graph convolution can be regarded as a special case of hypergraph convolution with small differences in the graph Laplacian matrix (as illustrated in [Bai et al., 2021]). Empirically this leads to a minor performance difference between HyperSCI-G and HyperSCI; 2) When k increases, the performance of ITE estimation from both methods are gradually improved, but such an improvement becomes less significant when k is larger. Besides, we notice HyperSCI consistently outperforms HyperSCI-G and such a difference becomes larger as k increases, indicating its efficacy on modeling high-order interference especially on large hyperedges.

5 Conclusion

We study an important research problem of ITE estimation with the existence of high-order interference on hypergraphs. We identify and analyze the influence of high-order interference in causal effect estimation. To address this problem, we propose a novel framework HyperSCI, which estimates the ITEs based on representation learning. More specifically, HyperSCI learns the representation of confounders, models the high-order interference with a hypergraph neural network module, then predicts the potential outcomes for each instance with the learned representations. Extensive experiments conducted on semi-synthetic data based on real-world hypergraphs consistently validate the effectiveness of HyperSCI in ITE estimation under different interference scenarios.