Optimality, Accuracy, and Efficiency of an Exact Functional Test

Optimality, Accuracy, and Efficiency of an Exact Functional Test

Hien H. Nguyen, Hua Zhong, Mingzhou Song

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence
Main track. Pages 2683-2689. https://doi.org/10.24963/ijcai.2020/372

Functional dependency can lead to discoveries of new mechanisms not possible via symmetric association. Most asymmetric methods for causal direction inference are not driven by the function-versus-independence question. A recent exact functional test (EFT) was designed to detect functionally dependent patterns model-free with an exact null distribution. However, the EFT lacked a theoretical justification, had not been compared with other asymmetric methods, and was practically slow. Here, we prove the functional optimality of the EFT statistic, demonstrate its advantage in functional inference accuracy over five other methods, and develop a branch-and-bound algorithm with dynamic and quadratic programming to run at orders of magnitude faster than its previous implementation. Our results make it practical to answer the exact functional dependency question arising from discovery-driven artificial intelligence applications. Software that implements EFT is freely available in the R package 'FunChisq' (≥2.5.0) at https://cran.r-project.org/package=FunChisq
Keywords:
Machine Learning: Unsupervised Learning
Uncertainty in AI: Exact Probabilistic Inference
Data Mining: Theoretical Foundations
Machine Learning: Relational Learning