1
$\begingroup$

I'm not sure what the purpose of the o(1-o) in the back propagation algorithm achieves? I'm guessing it's related to using the sigmoid function on the output but I'd like to have a proper understanding of the math behind it. Thanks!

$\endgroup$

migrated from cstheory.stackexchange.com Mar 2 '14 at 12:22

This question came from our site for theoretical computer scientists and researchers in related fields.

  • 1
    $\begingroup$ What research have you done? Back propagation is covered well in many textbooks. Have you tried reading them? We expect you to do a significant amount of research on your own before asking. $\endgroup$ – D.W. Mar 2 '14 at 15:39
1
$\begingroup$

Backpropagation is a gradient descent algorithm for minimizing the squared error of the learned network on the data given. In order to minimize by following the gradient, you need to know where the gradient points, this is done using calculus.

Along the way of calculating the gradient, you have to find the derivative of the activation function. When you take the most common activation function, i.e. the logistic rule or sigmoid function $o(z) = \frac{1}{1 - e^{-z}}$ and you differentiate it with respect to the neurons' input $z$ then you get $do/dz = o(1 - o)$. Hence, you are correct, this is related to the use of the sigmoid function, and figuring out in which direction the gradient points.

If you want a more thorough description then consult these notes (pdf).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.