2
$\begingroup$

I have seen that in the material made by Andrew Ng about neural networks, he uses the following weights:

enter image description here

so when I replace the final values of h_theta(x) in the formula:

enter image description here

I got values near 0 and 1.

The problem that I have is when I use the values that are in the Tom Mithell book about Machine Learning, he uses the values w0=-0.8, w1=w2=0.5. I have performed the calculations and got the values of h_theta(x) as the following:

0.8,0.3,0.3,-0.2

but when I replace these set of values into the formula of the sigmoid function I got:

0.31,0.42,0.42,0.54

that does not differentiate too much about which values are near 0 and which near 1, something that did not occur with the values that Andrew Ng proposed for the weights. Am I missing something? in this case it would be better to select big values for the weights instead of small ones?

Any help for clarifying this is welcome

$\endgroup$
1
$\begingroup$

It's just that the logistic function reaches its saturated regime around +-4.6, as in the image. So, any absolute values of x much lower than this will give you results around 0.5. Increasing the absolute value of the weights (all of them) you'll get nearer the saturated region, resulting in outputs closer to 0 and 1. Nevertheless, if you use the step function at the output layer instead of the logistic, you'll get the correct values exactly for both sets of weights: 0 0 0 1.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.