Saliency Maps

A method that highlights the pixels that are relevant for the prediction of a neural network.

Gradient-only Methods

• Vanilla Gradient
• Grad-CAM

Methods that indicate pixel relevance by how a single pixel would change the prediction based on gradients. The larger the absolute value of the gradient, the stronger the effect of a change of this pixel.

Using the gradient of the outputs only for determining the influence of a pixel on the output is not entirely correct as a CNN is a highly non-linear function. However, the idea is justified by assuming that the CNN can be approximated by a first-order Taylor expansion

$$Y\approx \nabla Y\cdot I+b$$

for an output $Y$ with gradient $\nabla Y=\frac{\partial Y}{\partial I}$, an image-pixel $I$, and a bias $b$.

The gradient $\nabla Y$ is found by Backpropagation.