I am currently trying to learn how the Delta Rule works in a neural network. So far I completely understand the concept of the Delta Rule, but the derivation doesn't make sense. My question is how is the Delta Rule derived and what is the explanation for the algebra. Currently I am writing equations to try to understand, they are as follows:
Desired = (Input * WeightD)
Actual = (Input * WeightA)
Error = (Input * WeightD) - (Input * WeightA)
Error = Input (WeightD - WeightA)
However, as stated Here, the ΔWeight is understood to be Error * Input, where as I derive it to be Error / Input. I fully understand the concept of correcting the weights in a neural network, however, I do not understand how the algebra and derivation of the Dela Rule work to re-adjust the weights to correctly represent the output. I will explain my current understanding to help you understand why I may be confused. Because the Input is constant whilst you attempt to find a correct weight, you can factor it out. This leads me to the equation that the Error is equal to the Input times the difference between the correct and incorrect weights. This is also very clear. However I do not understand where they go from here to get the equation to correct the weights. How is the required net change derived from this? How does the algebra behind simply Input * Error work? Thank you.