I'm not sure what the purpose of the o(1-o) in the back propagation algorithm achieves? I'm guessing it's related to using the sigmoid function on the output but I'd like to have a proper understanding of the math behind it. Thanks!
1 Answer
Backpropagation is a gradient descent algorithm for minimizing the squared error of the learned network on the data given. In order to minimize by following the gradient, you need to know where the gradient points, this is done using calculus.
Along the way of calculating the gradient, you have to find the derivative of the activation function. When you take the most common activation function, i.e. the logistic rule or sigmoid function $o(z) = \frac{1}{1 - e^{-z}}$ and you differentiate it with respect to the neurons' input $z$ then you get $do/dz = o(1 - o)$. Hence, you are correct, this is related to the use of the sigmoid function, and figuring out in which direction the gradient points.
If you want a more thorough description then consult these notes (pdf).