# max, min gradient and other terms in Neural Network

This link contains a demo that trains a Convolutional Neural Network on the MNIST digits dataset in browser.

I am not getting below terms- 1. max, min gradient in each layer. 2.max, min activation in layers

3.Forward time per example

4.Backprop time per example

1. Classification loss

2. L2 (a pooling mechanism??! but what this means?) Weight decay loss

What are the meanings of above terms according to given context (webpage demo)?

• You're asking about 6 different questions here. We generally prefer posts that ask a single question. Some of these questions may be off-topic here because they are specific to a particular implementation. Some of these can probably be answered by doing some research/searching on the name provided there. – D.W. Aug 6 '17 at 5:19

I suggest you to read an introduction to neural networks. I like neuralnetworksanddeeplearning.com, but I can also recommend chapter 4.3 of my bachelors thesis.

1. max, min gradient in each layer

The gradient is like the derivative for high-dimensional functions. Imagine it as an arrow on a landscape. The arrow points into the direction of the steepest ascent, we want to go into the other direction (where the error becomes smallest).

2.max, min activation in layers

The activation is $\sum w_i x_i$ (details and my BS thesis). Hence it is what each neuron returns, before the activation function is applied. Now you have many neurons per layer and hence a set of activations $A$. Hence you can take the min / max of it. I don't know why you would take the min, but the maximum might give you an indicator which neurons are "responsible" for which feature.

For CNNs, a complete feature map is usually considered to be one neuron (see my MS thesis for some more details). In your case, it simply seems to be the maximum feature returned (hence the brightest pixel; see input layer)

3.Forward time per example

Measure how long it takes to process the input of a neural network. (There is also the backward pass for the gradient)