Published on Tue Jul 21 2020

Pattern-Guided Integrated Gradients

Robert Schwarzenberg, Steffen Castle

Integrated Gradients (IG) and PatternAttribution (PA) are two established explainability methods for neural networks. We combine the two methods into a new method, Pattern-Guided Integrated Gradients. PGIG inherits important properties from both and passes stress tests that the originals fail.

0
0
0
Abstract

Integrated Gradients (IG) and PatternAttribution (PA) are two established explainability methods for neural networks. Both methods are theoretically well-founded. However, they were designed to overcome different challenges. In this work, we combine the two methods into a new method, Pattern-Guided Integrated Gradients (PGIG). PGIG inherits important properties from both parent methods and passes stress tests that the originals fail. In addition, we benchmark PGIG against nine alternative explainability approaches (including its parent methods) in a large-scale image degradation experiment and find that it outperforms all of them.

Thu Jun 17 2021
Computer Vision
Guided Integrated Gradients: An Adaptive Path Method for Removing Noise
Integrated Gradients (IG) is a commonly used feature attribution method for deep neural networks. IG often produces spurious/noisy pixel attributions in regions that are not related to the predicted class. To minimize the effect of this source of noise, we propose adapting the attribution path itself.
4
1
1
Fri Dec 20 2019
Machine Learning
When Explanations Lie: Why Many Modified BP Attributions Fail
A popular approach is to give a custom relevance score using modified rules. We find explanations of all mentioned methods, except for DeepLIFT, are independent of the parameters of later layers. We provide theoretical insights for this surprising behavior.
0
0
0
Tue May 16 2017
Machine Learning
Learning how to explain neural networks: PatternNet and PatternAttribution
DeConvNet, Guided BackProp, LRP were invented to better understand deep neural networks. We show that these methods do not produce the theoreticallycorrect explanation for a linear model. This is a cause for concern since linear models are simple neural networks.
0
0
0
Tue Sep 01 2020
Machine Learning
Learning explanations that are hard to vary
In this paper, we investigate the principle that `good explanations are hard to vary' in the context of deep learning. We show that averaging gradients across examples -- akin to a logical OR of patterns -- can favor memorization and patchwork solutions.
1
0
0
Wed Nov 15 2017
Computer Vision
Interpreting Deep Visual Representations via Network Dissection
Network Dissection is a method that provides labels for the units of their deep visual representations. The proposed method quantifies the interpretability of CNN representations by evaluating the alignment between individual hidden units and a set of visual semantic concepts.
0
0
0
Wed Oct 09 2019
Machine Learning
Explaining image classifiers by removing input features using generative models
Perturbation-based explanation methods often measure the contribution of an input feature to an image classifier's outputs by heuristically removing it via blurring, adding noise, or graying out. Instead, we propose to integrate a generative inpainter into three representative attribution methods.
0
0
0