Conditional Independent Test in the Presence of Measurement Error with Causal Structure Learning

1 Introduction

The conditional independence (CI) test aims to assess whether a correlation relation exists between two variables when conditioned on a set of other variables. It has been broadly applied across various fields like statistics, machine learning, and causal discovery [Zhou et al., 2020; Watson and Wright, 2021; Bouezmarni and Taamouti, 2014; Lundborg et al., 2022; Spirtes et al., 2000].

When the data model is presumed to be linear, the partial correlation test [Baba et al., 2004] is a well-known and effective tool for examining conditional independence relations among variables. Specifically, let Xi and Xj be two random variables and Xp be a vector, if the CI relation between Xi and Xj given Xp holds, i.e., Xi ⫫ Xk∣Xp, the partial correlation coefficient would be zero, denoted by ρXiXj .Xp = 0. Thus, one can examine whether the partial correlation coefficient is zero to discover the CI relations among observed variables. Obtaining these CI relations usually can be used to improve downstream tasks, such as constructing causal structures over observed variables, where the CI relations correspond to d-separation in the causal graphical model [Spirtes and Glymour, 1991; Spirtes et al., 2000].

∗Corresponding author.

However, due to uncertainties in the real environment, the variable of interest is often difficult to measure directly and instead collected under the existence of measurement errors [Totton and White, 2011; Fuller, 2009; Kelly, 2007]. An example of measurement error is depicted in Fig. 1, where Xi, Xj, and Xk represent hidden variables of interest. When there exists some disturbing noise that directly causes the observed variable, one can only observe the variables ˜ Xi, ˜ Xj , and ˜ Xk (known as measured variables). Since the CI relations only hold on Xi ⫫ Xj∣Xk, in this case, the partial correlation coefficient ρ ˜ Xi ˜ Xj. ˜ Xk ≠ 0, implying the CI relations of Xi ⫫ Xj∣Xk can not be tested from ˜ Xi, ˜ Xj, and ˜ Xk (further details can be found in Example 2). Such a problem will lead to uninformative relations discovery or incorrect causal structure learning. Therefore, it is necessary to investigate how to learn the CI relations among hidden variables only from their measured variables.

Measurement error has been a topic of increasing interest in causal discovery. For instance, [Zhang et al., 2018] introduces a method for inferring causal structures in the presence of measurement error by leveraging non-Gaussianity, showing that the causal structure can be identified up to an ancestral ordered grouping (AOG) equivalence class. Similarly, [Yang et al., 2022] exploits the sparsity of the mixing matrix to propose a method for identifying linear latent variable models affected by measurement error. These identifiability results mainly rely on overcomplete independent component analysis (ICA), which, despite its theoretical appeal, often encounters computational challenges and risks convergence to local optima. Recently, [Dai et al., 2022] introduced the TIN condition to capture causal structure patterns in linear non-Gaussian models, proving that the partially causal order can be identified even in the presence of measurement error. While this approach effectively verifies d-separation relations (i.e., conditional independence) among hidden variables, it imposes structural constraints, such as the requirement that each hidden variable must have at least two observed variables as children. This assumption significantly restricts the applicability of the TIN condition in scenarios involving measurement error. Overall, existing methods struggle to efficiently identify conditional independence (CI) relations among latent variables solely from the observed variables without imposing additional assumptions. As shown in Fig. 1, for instance, existing methods may fail to effectively test the Xi ⫫ Xj ∣Xk by only using measured variables ˜ Xi, ˜ Xj and ˜ Xk .

In this paper, we address the problem of testing conditional independence (CI) relations among hidden variables only from their measured variables, under the linear nonGaussian measurement error model. We find out a sufficient condition-namely, the partial Gaussian noise assumption, where the noise term of the observed variables follows a Gaussian distribution-that enables the identification of dseparation relations from the high-order statistics of the hidden variables. Specifically, we first establish an equivalence between partial correlation and the rank deficiency of the correlation coefficient matrix. We then prove that rank constraints on the covariance matrix can be generalized to the higher-order cumulant matrix, enabling the identification of d-separation relations among latent variables. Based on this theoretical result, we propose a structure learning algorithm, the PC-ME algorithm, to infer causal structures among latent variables from their measured variables. Furthermore, we show that the PC-ME algorithm can identify the causal structure up to a Markov equivalence class.

Our contributions are summarised as follows:

• We generalize partial correlation to the case of measurement error and study the properties of rank deficiency of the higher-order cumulant matrix, under the linear nonGaussian measurement model and the partial Gaussian noise assumption.
• We propose an efficient algorithm to infer causal structures among hidden variables up to a Markov equivalence class by leveraging the properties of rank deficiency in the higher-order cumulant matrix.
• We empirically demonstrate that the proposed algorithm can asymptotically recover the correct causal structure under mild assumptions.

2 Linear Non-GaussianMeasurement Error Model

In this paper, we focus on the problem of testing conditional independence in the presence of measurement error, specifically within the framework of linear non-Gaussian models. Our theoretical results are built upon causal graphical models. In a causal graph, we use Pa(Vi) = {Vj∣Vj → Vi}, Ch(Vi) = {Vj∣Vi → Vj} to denote the set of parents and children of Vi, respectively. Due to space limitation, some graphical notations, such as d-separation and V-structure, are used here without explicit definitions, which can be found in the standard literature [Spirtes et al., 2000].

We start with the definition of the linear non-Gaussian model. Let X = {X1,⋯,Xn} denote the variable set of interest, in the linear non-Gaussian model, the data generation process for Xi ∈ X is given by the following equation:

where PaXi is the parent set of Xi, bij is the causal effect fromXj to Xi, and ni represents noise terms of Xi that follows non-Gaussian distribution. Without loss of generality, we assume all noise terms are independent of each other and the mean of noise is zero.

In real-world scenarios, variables of interest are often difficult to observe directly (commonly referred to as unobserved or hidden variables) and are instead measured with additive measurement error. To formalize the existence of measurement error, we present the linear non-Gaussian Measurement error Model (LiNGMME) as follows.

Definition 1 (LiNGMME). Let X be the hidden variable of interest that follows the linear non-Gaussian model, and ˜Xbe the measured variable set of X, we say ˜X follows the LiNGMME model if for ˜ Xi, ˜ Xi ∈ ˜X, it is generated by:

where Xi ∈ X is hidden variable, βXi→ ˜ Xi is disturbing strength fromXi to ˜ Xi, and ei are measurement error term.

Recently, the linear non-Gaussian model with measurement error has been extensively studied, such as [Yang et al., 2022; Zhang et al., 2018; Dai et al., 2022]. These works usually assume that all noise term, including those associated with hidden variables and measured variables, follow a non-Gaussian distribution. Under this assumption, the causal structure can be identified up to an Ancestral Ordered Grouping (AOG) equivalence class [Zhang et al., 2018]. In this paper, we explore a different condition where the noise term of the measured variables follows a Gaussian distribution, which enables the identifiability of conditional independence relations among hidden variables.

Condition 1 (Partial GaussianCondition). In LiNGMME, the noise term of the measured variable ei follows a Gaussian distribution.

In general, Condition 1 is reasonable and holds in most cases, as the Gaussian distribution is ubiquitous and commonly observed in many real-world scenarios, as discussed in [Totton and White, 2011; Fuller, 2009; Kelly, 2007].

Objective. In this paper, assuming Condition 1 holds, our goal is to investigate how to identify the conditional independence (CI) relations among hidden variables X only using their measured variables ˜X, under the LiNGAMME.

3 Conditional Independence Test Under Measurement Error

To achieve our goal, in this section, we provide an algebraic solution for identifying the conditional independence (CI) relations among hidden variables. First, we revisit partial correlation testing, a well-known method for testing CI, and explain why it fails in the presence of measurement error (Sec. 3.1). Next, we derive an equivalent expression for partial correlation from an algebraic perspective, specifically through the rank deficiency of the covariance matrix, and demonstrate how this can be extended to rank constraints on the higherorder cumulant matrix. By exploring these properties, we prove that the CI relations among latent variables can be identified (Sec. 3.2).

3.1 Partial Correlation Test

In a linear model, such as the linear Gaussian model (i.e., noise term follow Gaussian distribution in Eq. (1)), partial correlation is a commonly used tool for testing conditional independence, and is specifically defined as follows:

Definition 2 (Partial Correlation). The partial correlation coefficient of random variables Xi and Xj given Xp, is de-

where ρXiXj , ρXiXp , and ρXjXp represent the correlation coefficients of Xi and Xj, Xi and Xp, Xj and Xp, respectively

Specifically, CI relations can be examined by checking whether the partial correlation coefficient is zero. For instance, consider the structure in Fig. 1, where the CI relation Xi ⫫ Xj∣Xk holds. This can be tested by checking whether the partial correlation coefficient ρXiXj⋅Xk is zero. However, in the presence of measurement error, only ˜Xi, ˜Xj and ˜Xk are observed. In this case, the partial correlation test cannot reliably identify the CI relations among the latent variables Xi and Xj . Next, we give an example to illustrate this.

Illustrative Example

Consider the graph in Fig. 1, where Xi, Xj and Xk are hidden variables and ˜ Xi, ˜ Xj, and ˜ Xk are observed variables that are measured with measurement error. According to the definition of LiNGAMME, the data generation processes are

To construct the CI test using partial correlation, it is sufficient to check whether the numerator is zero. Specifically, each term of the numerator of the partial correlation coefficient between ˜ Xi and ˜ Xj given ˜ Xk is as follows.

Without loss of generality, assume that all observed variables are standardized, i.e., have unit variance. If the numerator of the partial correlation coefficient is zero, then we have

which does not hold in most cases. Note that if the above holds, it implies that either no measurement error exists or the measurement error can be ignored. There are two factors that prevent the partial correlation test from working: one is the disturbing strength from Xk to ˜Xk, and the other is the measurement error noise ek (or, ˜Xk). Generally speaking, without further constraints, the partial correlation test cannot be used in the presence of measurement error.

3.2 Rank Deficiency of High-order Cumulants Matrix

The above example illustrates why partial correlation does not work in the presence of measurement error: there exists disturbing strength βXk→ ˜Xk and an additional noise term ek, which causes the vanishing test to fail. To tackle this issue, fortunately, we have found that these unexpected influences can be mitigated under Condition 1, by appropriately utilizing the properties of higher-order cross-cumulants.

Before introducing the main result, we first provide an equivalent representation of partial correlation from the algebraic perspective, used as the building block for later results. We begin with the following example.

Example 1 (Equivalent representation of partial correlation). Consider the graph in Fig. 1, where the CI relationship Xi ⫫ Xj∣Xk holds. This CI relation can be tested by checking whether the partial correlation coefficient ρXiXj⋅Xk is zero, i.e., ρXiXj - ρXiXk ⋅ ρXkXj = 0 holds. Without loss generality, assume Xi,Xj and Xk are all standardized, so that ρXkXk = 1. The vanishing of the partial correlation can then be rewritten as:

Such an equality can be transferred to the determinant of the correlation coefficient matrix between {Xi,Xk} and {Xj,Xk}, as shown below:

which means that the correlation coefficient matrix is not full rank, i.e., rank deficiency.

In our work, we refer to this algebraic property as a "rank constraint," following the terminology used in the original literature [Sullivant et al., 2010; Huang et al., 2022]. We further present the following lemma to establish the connection between the CI relation and the rank constraints of the coefficient matrix. In other words, the relationship between partial correlation and the rank deficiency of the coefficient matrix.

Lemma 1 (Rank constraints of covariance matrix [Sullivant et al., 2010]). AssumeXi,Xj andXp follow the linear Gaussian model, and let Y = (Xi, Xp) and Z = (Xj , Xp) represent two vectors, then Xi ⫫Xj∣Xp if and only if the rank of correlation coefficient matrix between Y and Z is ∣Xp∣.

Lemma 1 provides an equivalent formalization of the partial correlation test, where the rank constraint of the covariance matrix between two vectors holds, with the intersection set of the two vectors serving as the conditional set in the CI relation. It is important to note that, when all variables are standardized, the rank constraints of the coefficient matrix are equivalent to those of the covariance matrix.

Recall the illustrative example in Sec 3.1. Let Y = { ˜Xi, ˜Xk} andZ = { ˜ Xj, ˜Xk}, assuming all variables are standardized. In most cases, the correlation coefficient (covariance) matrix between Y and Z has full rank due to the influence of measurement error. Under Condition 1, the key idea to address this issue is the fact that the higher-order cross-cumulants of Gaussian noise are zero. Based on this observation, one may wonder whether the rank constraints of the covariance matrix can be extended to a higher-order cross-cumulants matrix, thereby removing the influence of the Gaussian noise term of the measured variables. Before delving into this fact, we first revisit the definition of cumulants and some fundamental properties.

Definition 3 (k-th order joint cumulant tensor). The kth order joint cumulant tensor of a random vector X = [X1, ...,Xn]T is the k-way tensor T (k) X inRn×⋯×n ≡ (Rn )k whose entry in (i1, ..., ik) is the joint cumulant:

where the sum is taken over all partitions (B1, . . . ,BL) of the multiset {i1, ..., ik}.

Remark 1. Consider k = 4, one can see that the four-order cumulant cum(Xi1 ,Xi2 ,Xi3 ,Xi4 ) is defined as

That is, the fourth-order cumulant is not equal to the fourth-order moment. Specifically, all non-fourth-order interaction terms are absent in the fourth-order cumulant.

One can see that the second-order cumulant is the covariance, and higher-order cumulants can be viewed as an extension of covariance, where certain properties, such as additivity, are preserved. This motivates us to further investigate the matrix of cross-cumulants, which corresponds to the 2D slice of the joint cumulant tensor.

Definition 4 (2D slice of joint cumulant tensor). For a random vector X with k-th order joint cumulant tensor T (k) X where k ≥ 2 and k is any even, denote its 2D matrix slice of k-th order joint cumulant tensor as C (k) , where

In particular, when k = 2, for a random vector X, C (2) X is the covariance matrix ΣX. Similar to Lemma 1, one can construct the cumulant matrix between two vectors Y and Z, as shown in the following example.

Example 2 (K-order cumulant matrix). For two vertices sets Z = {X1,X2,⋯,Xn},Y = {Xn+1,Xn+2,⋯,X2n}, let k ≥ 2, the k-order cumulant matrix between Z and Y is

Next, we will show that in the LiNGAMME, the CI relations among hidden variables can be tested by checking the rank constraints over the cumulant matrix of observed variables, as shown in Theorem 1.

Theorem 1 (Rank constraints of cumulant matrix). For hidden variables Xi, Xj, and a vector Xp, assume that their observed variables ˜ Xi, ˜ Xj, ˜ Xp follow the LiNGMME and any k-order cumulant among them exists. Let Y = { ˜ Xi, ˜ Xp} and Z = { ˜ Xj, ˜ Xp}. Then, under Condition 1, Xi ⊥⊥ Xj∣Xp generic hold if and only if the rank of k-order cumulant matrix between Z and Y is ∣Xp∣, i.e., Rank(C kY;Z) = ∣Xp∣.

Remark 2. One can see that Lemma 1 is a special case (when k = 2) of Theorem 1. The key difference is that, while CI relations cannot be identified using the rank of the covariance matrix, they can be tested by Theorem 1 under Condition 1.

We further provide an example to show how CI relations can be tested using the rank constraints of the cumulant matrix, even when the partial correlation test fails.

Example 3 (Testing CI relations using Theorem 1). Continuous the illustrative example in Sec. 3.1. Let Y = { ˜Xi, ˜Xk} and Z = { ˜Xj, ˜Xk}, assuming the 4-order cross-cumulant exists. Then the cumulants matrix between Y and Z is

where

One can see that C (4) Xi ,Xj C (4) Xk ,Xk - C (4) Xi ,Xk C (4) Xj,Xk = 0, which implies that the cumulant matrix C (4) Y;Z is rank deficiency, i.e., Rank(C (4) Y;Z) = 1. By Theorem 1, we can infer that Xi ⫫ Xj ∣Xk.

Intuitively, the reason why CI relations can be tested by checking the rank constraints of the cumulant matrix is that the cumulant matrix excludes any information from Gaussian measurement error noise due to the property that higher-order cumulants of Gaussian distributions are zero, e.g., E(e4k) = 0 in C (4) Xk,Xk . This result can be directly applied to the linear non-Gaussian causal model, providing an alternative method for CI testing.

Corollary 1 (Rank constraints in linear non-Gaussian model). Let Xi, Xj be two random variables and Xp be a vector in the linear non-Gaussian model. Suppose any korder cross-cumulant exists. Then, Xi ⊥⊥ Xj∣Xp generic holds if and only if the rank of any k-order cumulant matrix between vectors Y = (Xi, Xp) and vectors Z = (Xj , Xp) is ∣Xp∣, i.e., Rank(C kY;Z) = ∣Xp∣, where ∣Xp∣ are the dimension size of the vector.

In other words, Corollary 1 shows that if there is no measurement error, Lemma 1 generally holds and can be viewed as a special case of the rank constraints of the cumulant matrix, so that Rank(C 2Y,Z) = Rank(C kY,Z) with k ≥ 2. Practically, in the linear non-Gaussian model, one can enhance the statistical robustness of the CI test by summarizing the results across multiple k-order cumulants matrices.

Example 4. Let k = {4, 6, 8}, Y = { ˜Xi, ˜Xk} and Z = { ˜Xj, ˜Xk}. If we obtain results from observational data such that Rank(C (4) Y,Z) = 1, Rank(C (6) Y,Z) = 1, and Rank(C (8) Y,Z) = 2, we can infer that Xi ⫫ Xj∣Xk, because most of the results tend to support the rank deficiency of the cumulant matrix.

Moreover, in practice, an important problem is how to test the rank constraints of the cumulant matrix using empirical data. To address this, we propose the following hypothesis test.

Test the rank of Cumulant Matrix We test the rank of cumulant by leveraging CR statistic [Schkoda and Drton, 2023; Robin and Smith, 2000], which is formed to test the null hy-

where ˆ C be an asymptotically normal estimator of C and the null distribution of the CR statistic may be asymptotically approximated by a weighted sum of independent χ21 random variables, the weights being determined by the asymptotic covariance matrix of ˆ C.

4 Application to Causal Structure Learning

Conditional independence (CI) tests are widely used in causal discovery, where that the true causal structure of n random variables can be represented by a directed acyclic graph

(DAG) G. In this section, we apply the proposed CI testing tool to the causal discovery task, presenting an algorithm that can infer causal structures from observational data in the presence of measurement error. To ensure the asymptotic correctness of the causal discovery methods, some necessary causal assumptions are required.

Assumption 1 (Causal Markov condition). The joint distribution P satisfies all CIs that are imposed by the true causal graph (generating process of the data).

Assumption 2 (Rank Faithfulness [Spirtes, 2013]). Let a distribution P be (linearly) rank-faithful to a directed acyclic graph G if every rank-constraint on a sub-cumulants matrix that holds in P is entailed by every free-parameter linear structural model with path diagram equal to G.

Note that the causal Markov condition and faithfulness assumption are commonly used in the constraint-based causal discovery algorithm, such as PC algorithm [Spirtes et al., 2000]. Traditionally, the PC algorithm recovers the graph structure by leveraging the (conditional) independence relations identified in the data. However, it is well known that small mistakes early in the algorithm (e.g., failing to detect an independence relation) can propagate and lead to significant errors in the resulting DAG. Consequently, the performance of such methods heavily depends on the accuracy of the (conditional) independence testing procedures.

In the LiNGAMME framework, as shown in the illustrative example in Sec. 3.1, the conditional independence test becomes unreliable in the presence of measurement error, which can lead to incorrect results produced by the PC algorithm. To address this issue, by replacing the conditional independence test in the PC algorithm with our proposed tool, we present the PC algorithm with measurement error (abbreviated as PCME), as shown in Algorithm 1.

As shown in Algorithm 1, we adopt the standard framework of the PC algorithm to learn the causal structure from observed variables in the LiNGAMME framework. Specifically, we initialize the complete undirected graph G on the set of hidden variables X. Then, we iteratively check whether each pair of hidden variables, Xi and Xj, are conditionally independent given a set of hidden variables Xp. This is achieved by testing the rank constraint of the cumulant matrix among the observed variables. Formally, Xi ⫫ Xj ∣ Xp holds if and only if Rank(C kY;Z) = ∣Xp∣, where Y = { ˜Xi, ˜Xp} and Z = { ˜Xj, ˜Xp}. If Xi and Xj are conditionally independent given a conditional set, the edge between Xi and Xj in G is removed. By repeating this procedure, we obtain the skeleton of the hidden variables. Simultaneously, we record the corresponding CI test results, including the conditional set of Xi and Xj in the d-separated set Sepset(Xi,Xj), which is useful for inferring causal directions and reducing redundant tests in subsequent steps. Next, we orient the causal directions by detecting V-structures, i.e., checking whether a local structure Xi - Xk - Xj forms a V-structure. If it does, we orient it as Xi → Xk ← Xj. Finally, we apply Meek's rules [Meek, 1995] to infer additional causal directions and output a partial DAG (PDAG) of the hidden variables with respect to the observational data.

Algorithm 1 PC algorithm with measurement error (PC-ME)

Require: Observational dataset ˜X = { ˜ X1, . . . , ˜ Xm} Ensure: A partial DAG G.

Let ˜ Xi denote as its corresponding hidden variable Xi, Xi ∈ X;

Initialize the complete undirected graph G over the hidden variable set X;

for ∀Xi,Xj ∈ X and adjacent in G do

if ∃Xp ⊆ X \ {Xi,X)j} and (∣Xp∣ < k) such that Rank(C kY;Z) = ∣Xp∣ with Y = { ˜ Xi, ˜ Xp}, Z = { ˜ Xj, ˜ Xp} then

end if end for

for ∀Xi,Xj,Xk ∈ X such that the pair Xi, Xk and the pair Xj , X)k are adjacent in G but the pair Xi, Xj are not adjacent in G do

end if

end for

while at least one edge can be oriented do

if ∃Xi → Xj, Xj and Xk are adjacent, Xi and Xk are not adjacent and there is no arrowhead at Xj then orient Xj - Xk as Xj → Xk

end if

if ∃Xi and Xj, there is a directed path from Xi to Xj and an edge between Xi and Xj then orient Xi - Xj as Xi → Xj

end if

end while

Return a partial DAG of hidden variables;

Complexity analysis. The complexity of the PC-ME algorithm for a hidden variable causal structure graph G is bounded by the largest degree in G. Let k be the maximal degree of any vertex and let n be the number of vertices. Then in the worst case, the number of conditional independence tests among hidden variables required by the algorithm is bounded by

Since the PC-ME algorithm relies only on the constraints of conditional independence (CI), it generally identifies the causal structure only up to Markov equivalence classes, which are sets of graphs that represent exactly the same independences and CIs. Thus, we next give the identifiability results of the PC-ME algorithm.

Theorem 2 (The correctness of algorithm). Suppose the data generation process follows the LiNGAMME, and the causal Markov condition, rank faithfulness assumption, and Condition 1 hold. Given a sufficiently large sample size, the PC-ME algorithm asymptotically outputs the Markov equivalence classes of the hidden variables.

5 Simulation Experiments

In this section, we evaluate the practical performance of the proposed algorithm using synthetic data. First, we examine the performance in the CI test based on the rank constraints of cumulant matrices. Then, we verify the correctness of the PC-ME algorithm in the task of causal structure learning.

5.1 Performance of Conditional Independence Test

We conducted an experiment to verify the performance of the conditional independence test under the LiNGAMME. We consider the case whether Xi and Xj are conditional independence given by Xp, i.e., Xi ⫫ Xj∣Xp, where Xp = {X1, ..., Xm}. We adjust the dimension m of Xp to show that the proposed method is able to deal with the complexity case of conditional independent, ranging from 1 to 7. Specifically, we first generated the independent vector Xp, then Xi and Xi were generated as β⊺Xp + ε , where β is causal strength vector, which is generated uniformly in [-2, -0.5]∪[0.5, 2], ε is the noise term that follow the uniform distribution with U(-1, 1). Finally, we generate the observed variable according to the definition of LiNGAMME.

We compare our method with the partial correlation [Baba et al., 2004]- the well-known CI test method in linear models. We use Type I and Type II errors as evaluation metrics (for more details, refer to the appendix) to investigate how the size of the conditional set impacts performance. To see the performance of Type II, not all elements in Xp are the conditional set of Xi and Xj. We randomly select the independent variable into Xp with ranging 1 to 3. In our simulations, the sample size is fixed at [1000, 5000], and the significance level is set to [0.05, 0.01] for all methods.

The results are shown in Fig. 2, one can see that the probabilities of Type I errors closely align with the significance levels for our method, and meanwhile, the Type I error is close to significance level. These results show that our CI test tool can tackle the CI test under the existence of measurement error. Although with the increase in the dimensionality of Xp, the performance of our method noticeably decreases. This decrease is reasonable because the accuracy of the rank test decreases with the higher dimension matrix [Schkoda and Drton, 2023]. Furthermore, all of baseline methods exhibit poor performance in terms of Type I&II error, indicating the existing methods are not capable of handling the CI test in the presence of measurement error.

5.2 Performance of Causal Structure Learning

To evaluate the effectiveness of the proposed algorithm in causal structure learning, we employ the PC-ME algorithm on the random graph under the LiNGAMME model. We then compare its performance with that of the original PC algorithm, where CI relations are tested by partial correlation [Spirtes et al., 2000], and latent PC algorithm (abbreviated as LPC) [Chen et al., 2024], which allow to test the CI relations among hidden variables form measured variables.

Firstly, we construct the latent structure by randomly generating a Directed Acyclic Graph (DAG) [Kalisch and B¨uhlman, 2007], which can be represented as an n × n adjacency matrix, where n denotes the dimensionality of the latent variables. This matrix encodes the causal relationships among the hidden variables, with each element representing the causal strength between them. The hidden variable data is then generated from this DAG, with causal strength constrained to the interval [-2, -0.5] ∪ [0.5, 2], while the error term ni of each hidden variable Xi is sampled from either (i) a uniform distribution ni ∼ U(0.2, 1) (here four-order cumulant matrix is used in our method), or (ii) a standard exponential distribution. Subsequently, we generate observed variables corresponding to each hidden variable based on the definition of the LiNGAMEE. For instance, ˜ Xi = βXi→ ˜Xi Xi+ei, where ˜Xi is the observed variable of hidden variable Xi, βXi→ ˜Xi is disturbing strength, and ei is the corresponding noise term that follows the Gaussian distribution.

We simulate observational data under different settings and apply our algorithm along with baseline methods to these datasets. This allows us to assess how the performance varies across different scenarios. We control the following parameters: (1) varying the number of latent variables D = {5, 10, 15}, (2) adjusting the sample sizes {3k, 5k, 10k}. Additionally, we set the sparseness parameter when generating the DAG to s = 2/(k-1), where k represents the number of hidden variables, ensuring an average of two neighbors for each hidden variable [Cui et al., 2018]. Moreover, since the output of the PC algorithm is a partial DAG, we utilize precision, recall, and F1 score as metrics. In our analysis, the significance level of all methods is set to α = 0.01.

The experiment result of learning the causal skeleton is shown in Table 1 and Table 2. Our method outperforms all of the baseline methods, particularly in terms of precision and F1 score. This implies that the causal structure of the hidden variable estimated by our method is correct in most cases. Besides, one can see that all of the baseline methods exhibit poor F1 scores, indicating that existing methods are not able to handle scenarios involving measurement errors effectively. Due to space limitations, additional details on the experimental results are provided in the appendix.

6 RelatedWorks

Numerous methods exist for conditional independence (CI) testing and causal discovery with latent variables. For CI testing, there are the residual-based methods [Zhang et al., 2022; Zhang et al., 2019; Chen et al., 2024] and kernel-based approaches [Zhang et al., 2011; Doran et al., 2014]. For the task of causal discovery with latent variables, there are secondorder-based ones [Silva et al., 2006; Chen et al., 2022; Huang et al., 2022; Xie et al., 2023], higher-order statistics-based approaches [Xie et al., 2022; Xie et al., 2024; Jin et al., 2024; Chen et al., 2023]. While effective for CI testing and latent causal discovery, these methods do not account for measurement error, leaving a key gap in existing approaches.

7 Conclusion

We study the problem of testing conditional independence among observed variables in the presence of measurement error. In the LiNGAMEE model, by incorporating the higherorder cumulant, we present the rank constraint of the cumulant matrix, a novel CI tool capable of accommodating measurement errors. We further apply the proposed tool to the task of causal structure learning and present a practical algorithm PC-ME. The theoretical tool elegantly extends the application scope of partial correlation to account for measurement errors, and the proposed algorithm offers a solution for investigating and addressing measurement errors in the analysis. How to relax the model assumptions, e.g., allowing non-linear relations, would be interesting future directions.

Acknowledgements

This research was supported in part by the National Natural Science Foundation of China under grants No. 62476163 and U24A20233, and the Guangdong Basic and Applied Basic Research Foundation under grant number 2023B1515120020. We appreciate the comments from anonymous reviewers, which greatly helped to improve the paper.