arXiv:1708.08296v1 [cs.AI] 28 Aug 2017 ABSTRACT were crucial - PDF document

EXPLAINABLE ARTIFICIAL INTELLIGENCE: UNDERSTANDING, VISUALIZING AND INTERPRETING DEEP LEARNING MODELS Wojciech Samek 1 , Thomas Wiegand 1 , 2 , Klaus-Robert M¨ uller 2 , 3 , 4 1 Dept. of Video Coding & Analytics, Fraunhofer Heinrich Hertz Institute, 10587 Berlin, Germany 2 Dept. of Computer Science,Technische Universit¨ at Berlin, 10587 Berlin, Germany 3 Dept. of Brain & Cognitive Engineering, Korea University, Seoul 136-713, South Korea 4 Max Planck Institute for Informatics, Saarbr¨ ucken 66123, Germany arXiv:1708.08296v1 [cs.AI] 28 Aug 2017 ABSTRACT were crucial factors to success. Today’s machine learning- based AI systems excel in a number of complex tasks ranging With the availability of large databases and recent improve- from the detection of objects in images [14] and the under- ments in deep learning methodology, the performance of AI standing of natural languages [8] to the processing of speech signals [10]. On top of that, recent AI 1 systems can even out- systems is reaching or even exceeding the human level on an increasing number of complex tasks. Impressive examples play professional human players in difficult strategic games of this development can be found in domains such as image such as Go [34] and Texas hold’em poker [28]. These im- classification, sentiment analysis, speech understanding or mense successes of AI systems, especially deep learning strategic game playing. However, because of their nested models, show the revolutionary character of this technology, non-linear structure, these highly successful machine learn- which will have a large impact beyond the academic world ing and artificial intelligence models are usually applied in and will also give rise to disruptive changes in industries and a black box manner, i.e., no information is provided about societies. what exactly makes them arrive at their predictions. Since However, although these models reach impressive predic- this lack of transparency can be a major drawback, e.g., in tion accuracies, their nested non-linear structure makes them medical applications, the development of methods for visual- highly non-transparent, i.e., it is not clear what information izing, explaining and interpreting deep learning models has in the input data makes them actually arrive at their decisions. recently attracted increasing attention. This paper summa- Therefore these models are typically regarded as black boxes . rizes recent developments in this field and makes a plea for The 37th move in the second game of the historic Go match more interpretability in artificial intelligence. Furthermore, between Lee Sedol, a top Go player, and AlphaGo, an artifi- it presents two approaches to explaining predictions of deep cial intelligence system built by DeepMind, demonstrates the learning models, one method which computes the sensitiv- non-transparency of the AI system. AlphaGo played a move ity of the prediction with respect to changes in the input and which was totally unexpected and which was commented on one approach which meaningfully decomposes the decision by a Go expert in the following way: in terms of the input variables. These methods are evaluated on three classification tasks. “It’s not a human move. I’ve never seen a human play this move.” (Fan Hui, 2016). Index Terms — Artificial intelligence, deep neural networks, black box models, interpretability, sensitivity analysis, layer- Although during the match it was unclear why the system wise relevance propagation played this move, it was the deciding move for AlphaGo to win the game. In this case the black box character of the AlphaGo did not matter, but in many applications the impos- 1. INTRODUCTION sibility of understanding and validating the decision process of an AI system is a clear drawback. For instance, in medical The field of machine learning and artificial intelligence has diagnosis it would be irresponsible to trust predictions of a progressed over the last decades. A driving force for this black box system by default. Instead every far reaching de- development were earlier improvements in support vector cision should be made accessible for appropriate validation machines and more recent improvements in deep learning by a human expert. Also in self-driving cars, where a sin- methodology [22]. Also the availability of large databases gle incorrect prediction can be very costly, the reliance of the such as ImageNet [9] or Sports1M [17], the speed-up gains model on the right features must be guaranteed. The use of obtained with powerful GPU cards and the high flexibility of explainable and human interpretable AI models is a prereq- software frameworks such as Caffe [15] or TensorFlow [1] uisite for providing such a guarantee. More discussion on the This work was supported by the German Ministry for Education and Re- 1 The terms artificial intelligence and machine learning are used synony- search as Berlin Big Data Center BBDC (01IS14013A). We thank Gr´ egore Montavon for his valuable comments on the paper. mously.