Evaluating and Aggregating Feature-based Model Explanations

Evaluating and Aggregating Feature-based Model Explanations

Umang Bhatt, Adrian Weller, José M. F. Moura

Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence
Main track. Pages 3016-3022. https://doi.org/10.24963/ijcai.2020/417

A feature-based model explanation denotes how much each input feature contributes to a model's output for a given data point. As the number of proposed explanation functions grows, we lack quantitative evaluation criteria to help practitioners know when to use which explanation function. This paper proposes quantitative evaluation criteria for feature-based explanations: low sensitivity, high faithfulness, and low complexity. We devise a framework for aggregating explanation functions. We develop a procedure for learning an aggregate explanation function with lower complexity and then derive a new aggregate Shapley value explanation function that minimizes sensitivity.
Keywords:
Machine Learning: Explainable Machine Learning
AI Ethics: Explainability
Machine Learning: Interpretability