3
$\begingroup$

I am reading a very interesting paper on genetic algorithms which define neural networks. I am familiar with how a feedforward neural network operates, but then I came across this:

Recurrent connection.

Where node #4 goes back to connect to #5. I was wondering how this is handled? Does the state of node 4 get kept from the last timestep and applied to node 5 when it is time to calculate its activation?

Improve this question
$\endgroup$
1
  • $\begingroup$ We expect references to fulfill the minimal scholarly requirements and be as robust over time as possible. Please take some time to improve your post in this regard. We have collected some advice here. Thank you! $\endgroup$
    – D.W. ♦
    Commented Sep 19, 2016 at 4:41

1 Answer 1

Reset to default
2
$\begingroup$

When a recurrent network is calculated you can imagine the network is 'unrolled' out through time.

Unrolling an RNN

When visualized this way, you can see activation and loss can be calculated using the same method as a typical network.

So to clarify using the example you provided. Yes, the input to node #5 is the output from node #1, node #2, and the previous time steps node #4.

It is important to note; Typically the input and output to RNN's are expected to be time dependent. That is, for each time step t there is input t and output t. This makes RNN especially good at things like evaluating sequences. Although it is possible to keep the input and output equal throughout time.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.