0
$\begingroup$

I'm following Jeff heatons book 'Introduction to Neural Networks with Java'. To get node deltas, we need to calculate $f'(sum)$. In the very first row for Training Element #1, we need to compute $f'(1.13) \cdot 0.25$ which Heaton evaluates to $0.045$.

Using my calculator, I get $f'(1.13)=-0.13$ (derivative of sigmoid activation function), then I multiply by $0.25$ to get $-0.0325$. I've been trying to figure out for days how heaton does his calculation but no success yet. Kindly assist.

Improve this question
$\endgroup$
0
$\begingroup$

The derivative of a sigmoid is always positive. Here's my math: Sigmoid(1.13) = 0.7558

I computed this on Wolfram Alpha

The derivative of a sigmoid y=S(x) is y * (1-y), so 0.7558 * 0.2442 = 0.0451

Improve this answer
$\endgroup$
4
  • $\begingroup$ Hi,I think what you did is f '( Sigmoid(X) ).What the author did was f '(X)*0.25 and got 0.045 $\endgroup$ – Leroy Kayanda Apr 24 '14 at 5:04
  • $\begingroup$ @LeroyKayanda If your $f$ is the sigmoid, then MattD is correct. Certainly the fact that he got the same answer as the author is an indicator that his thinking parallels the book. I don't have that book and you haven't said enough to know. $\endgroup$ – Gene Apr 24 '14 at 23:54
  • $\begingroup$ Hi. Doing F ' ( Sigmoid(X) ) works with training element #1 but not with #2 in ( heatonresearch.com/wiki/Backpropagation ) . Here's the video that explains the calculation process youtube.com/watch?v=p1-FiWjThs8 $\endgroup$ – Leroy Kayanda Apr 25 '14 at 0:33
  • $\begingroup$ Hey, I finall got it @MattD is right. The derivative of sigmoid $f'(x)=f(x)*(1-f(x))$ Equivalent to $f'(x)$=sigmoid(x)*(1-sigmoid(x)) . What I was doing was $f'(x)=x*(1-x)$ Thanks for the help $\endgroup$ – Leroy Kayanda Apr 25 '14 at 5:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.