[2Q3-J-2] Machine learning: explainability, knowledge acquisition

[2Q3-J-2-04] Model-agnostic Explainer using Feature Patterns

〇Kohei Asano¹, Jinhee Chun¹, Takeshi Tokuyama¹ (1. Tohoku University)

Keywords:Interpretability, Machine Learning

Recently, high performance, though very complex, machine learning models have been proposed. Being able to interpret such black boxed decision making models is clearly critical for sensible tasks. In this paper, we propose a model-agnostic explanation method that details the prediction behavior of any models using feature patterns. Since the relationship between features is still unclear in the previous works, we focus on a combination of these features. We propose an algorithm which finds several minimal feature patterns that lead target prediction sufficiently using hill climbing search. In our experiments, we aim at measuring the faithfulness of our explanation, thus apply our method to sentiment analysis dataset and evaluate the faithfulness using two metrics: recall and precision. At last, we demonstrate the benefit of our method based on some image classification use cases with a black-box model.