# Back propagation in neural networks

I just finished watching these 3 Coursera videos on back propagation in neural networks. I get the idea of what we're trying to do, but I don't get how we achieve that by calculating error in each step as weight * cascaded error (eg. the formula at the top right of screen at 12:07 in the linked video). Let's say we start off with all the weights (theta) at zero. Wouldn't back propagation always calculate 0 error for everything, causing nothing to change?

• I suggest that you come up with the simplest example you can (of the smallest neural network possible), and draw that together with the current weights and a specific input and show us your calculations for why you think back propagation will calculate 0 error. Then add that to the question so we can help you better. It won't always calculate 0 error, but unless we can see your reasoning, it's hard for us to know where you went wrong or what your misunderstanding is. – D.W. Aug 19 '16 at 15:57
• Well, the formula is weight * error, so if weight is zero, then the formula will return 0 regardless of what the error at the next node was. – Atte Juvonen Aug 19 '16 at 22:24

When training neural networks, it is important to randomly initialize the parameters for symmetry breaking. One effective strategy for random initialization is to randomly select values for weights uniformly in the range [-x, x]. One effective strategy for choosing x is to base it on the number of units in the network. A good choice of x is x = sqrt(6) / sqrt(in + out) where in and out are the number of units in the layers adjacent to [the weights]. This range of values ensures that the parameters are kept small and makes the learning more efficient.