1
$\begingroup$

I am confusing on completing one epoch, I am using Single Layer Feed Forward neural Network approach. Lets suppose i have a data of OR Gate:

X1  X2   Target
0   0    0
0   1    1
1   0    1
1   1    1

Now we initialize weights (w1 and w2 with 0) and bias 1, it satisfy our first condition 0 0 0 with delta rule (T-O=0) but when we use same weight and same bias with 0 1 1 this row not satisfy with (w1 = 0 , w2 = 0, b= 1) so we change our weight and bias values, now my question is:

We changed value on 2nd row, now we have new weight values and bias value, do we need to check the third row which is 1 0 1 with new weight and bias value or with old one which is (w1=0, w2=0, b=1) ? and if we check 2nd row 3rd row and 4th row with new values and it satisfy 2nd 3rd and 4th row with new value, does this complete our 1st epoch ? but how because we didn't check the first row with new value.

$\endgroup$
3
$\begingroup$

We changed value on 2nd row, now we have new weight values and bias value, do we need to check the third row which is 1 0 1 with new weight and bias value or with old one which is (w1=0, w2=0, b=1) ?

The Perceptron processes one example at a time. The weights are immediately applied and subsequently updated (if the prediction is not correct).

if we check 2nd row 3rd row and 4th row with new values and it satisfy 2nd 3rd and 4th row with new value, does this complete our 1st epoch?

Applying the learning rule to each example in the training set is called an epoch.

Finishing an epoch doesn't mean that the perceptron converges to zero training error (it's typical to run multiple epochs).

For example, given:

(X1)--\
       \
        \ W1
         \
(X2)--W2--+-Y---O
         /
        / B
       /
 (1)--/

with:

$$Y = W_1 \cdot X_1 + W_2 \cdot X_2 + B$$

$$ O = \begin{cases} 0 & \text{if $Y \le 0$} \\ 1 & \text{if $Y > 0$} \end{cases} $$

$$\delta = Target - O$$

$$\Delta W = \epsilon \cdot \delta \cdot O$$

If $\epsilon = 1$ and $W_1 = W_2 = B = 0$:

X1  X2  Target   W1    W2     B     Y  O  Delta   W1'   W2'     B'
 0   0     0      0     0     0     0  0      0    0     0      0
 0   1     1      0     0     0     0  0      1    0     1      1  <-error
 1   0     1      0     1     1     1  1      0    0     1      1
 1   1     1      0     1     1     2  1      0    0     1      1
--------- END OF EPOCH 1 / NOT CONVERGED ------------------------
 0   0     0      0     1     1     1  1     -1    0     1      0  <-error
 0   1     1      0     1     0     1  1      0    0     1      0
 1   0     1      0     1     0     0  0      1    1     1      1  <-error
 1   1     1      1     1     1     3  1      0    1     1      1
--------- END OF EPOCH 2 / NOT CONVERGED ------------------------
 ...
--------- END OF EPOCH 3 / NOT CONVERGED ------------------------
 ...                                                              NO ERROR
--------- END OF EPOCH 4 / CONVERGED ----------------------------
$\endgroup$
  • $\begingroup$ Thank you for such explanation, my confusion is if we have lets suppose 0 1 1 and it not satisfy lets suppose our weight (w1=0.2, w2=0.2, bias=0.9), we update the weight for 0 1 1 , the new weight and bias must have to calculate delta = 0 ? What if i update weight for 0 1 1 and new weight and bias not give me delta 0 for 0 1 1 $\endgroup$ – ARG Oct 27 '17 at 12:16
  • $\begingroup$ The goal of the update is to adjust the weights so that they are better for the current example (if the example is presented twice in a row, the Perceptron should do a better job the second time). Correct classification of an example (delta=0) is not guaranteed immediately after the update. $\endgroup$ – manlio Oct 27 '17 at 12:58
  • $\begingroup$ You can take a look at A Course in Machine Learning - Chapter 4: it contains a clear (and simple) explanation of the Perceptron. $\endgroup$ – manlio Oct 27 '17 at 13:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.