1
$\begingroup$

I am reading about a method called guided backpropagation. https://ramprs.github.io/2017/01/21/Grad-CAM-Making-Off-the-Shelf-Deep-Models-Transparent-through-Visual-Explanations.html#deconv-and-guided-backprop.

As shown in the figure below, you essentially compute the derivates of the output with respect to the input image while zero-ing out negative gradients along the way. The website mentions

Negative gradients at a particular ReLU neuron, state that this neuron has a negative influence on the class that we are trying to visualize.

During backpropagation there are paths that have positive influence and some that have negative influence, and these end up cancelling out in a weird interference pattern, causing gradients to seem noisy. Whereas in Guided Backpropagation, we only keep paths that lead to positive influence on the class score, and supress the ones that have negative influence, leading to much cleaner looking images.

My question is why does the resulting image have pixels which are negative? Don't negative pixels indicate that the gradient of the output with respect to the pixel is negative? I.e. that pixel has a negative influence on the class score?

enter image description here

enter image description here

$\endgroup$
3
$\begingroup$

Guided backpropagation from the model output to the input of the first ReLU indeed results in non-negative values (as the gradient from ReLU output wrt. ReLU input is set to zero if it is negative). From the input of the first ReLU the gradient wrt. the input pixels can however become negative. Input images are mostly scaled to the range [-1,1] (e.g. by division by 127.5 and subtraction of 1 for Keras' pretrained models - that means roughly achieving zero mean and unit variance). If a negative input (pixel that is darker than the mean) contributes positively - via negative weights - to the output of the first convolutional layer the gradient for that pixel is negative. As negative gradients are only set to zero in guided backprop when passing through a ReLU and there is no ReLU between the input of the first ReLU and the overall input, guided backprop can result in negative values.

| cite | improve this answer | |
$\endgroup$
3
$\begingroup$

Kinda late but According to the gradcam paper (https://arxiv.org/abs/1610.02391), it says that negative gradient of the image will most likely contribute to other class.

We apply a ReLU to the linear combination of maps because we are only interested in the features that have a positive influence on the class of interest, i.e. pixels c whose intensity should be increased in order to increase y. Negative pixels are likely to belong to other categories in the image. As expected, without this ReLU, localization maps sometimes highlight more than just the desired class and achieve lower localization performance

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This doesn't answer the question on how values in the result of Guided Backpropagation can become negative although negative gradients are set to zero during backpropagation through ReLUs. $\endgroup$ – Dominik Friedmann Jun 16 at 19:36
0
$\begingroup$

The focus is not on negative pixels but on negative gradients. Negative gradients indicate that the direction of change in the input space decreases the probability of predicted as a certain class. Intuitively, it tells what this class is NOT about. For example, if you are looking for a cat in an image, there are plenty of ways to make the image NOT look like a cat. But that information is NOT useful and contributes to noises in the visualizations. Put another way, we are only interested in what makes the image look like the class of interest.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.