Wikipedia says:

The backward propagation of errors or backpropagation, is a common method of training artificial neural networks and used in conjunction with an optimization method such as gradient descent.

Ultimately, backward propagation is used to get the partial derivates of the weights and biases of the network, so that gradient descent can be used.

This means you end up with $\frac{\partial TotalError}{\partial weight_i}$ and $\frac{\partial TotalError}{\partial bias_i}$ at the end, for all weights and biases, but you don't actualy end up with any sort of specific error value for the neurons.

With that in mind, why is it called back propagation of errors, instead of backpropagation of error gradient or something similar?

  • 1
    $\begingroup$ It's just a name. I wouldn't look too much into it. The colloquial name is just backpropagation anyway. $\endgroup$ Mar 1 '17 at 0:56

It's actually the error for the neurons. The actual reason for derivation is this:

  • You find an error then you find the derivative of the error according to some differential equation.

  • Gradients you obtain using that derivative is basically to determine how much one single neuron in the network made mistake. Gradient at one neuron means if you update the weights or biases according to that you decrease the error network makes.

Backpropagation is the mathematical system for assesing how much one neuron made mistake so you can improve it. Definition is true in that sense.

  • $\begingroup$ I sort of see what you are saying. This page walks through backprop step by step mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example . dError/dOutput is calculated as (out-target), and then you calculate the derivatives of the weights using the chain rule starting from there. I guess it is propagating the error backwards after all. $\endgroup$
    – Alan Wolfe
    Mar 1 '17 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.