Usually, when I program recurrent neural networks, I use a loop for each neuron to figure out it's state. What I realized with this is that in this case, no neuron gets any feedback. They just pump their outputs to the next neuron in the cycle. My thoughts on how to counter this is by having neurons in two different time states. The "second" before and the "second" now. What each hidden neuron takes as an input from the other neurons is their activations from the "second" before to calculate the value of it's activation for that specific "second" which after they are all cycled, is stored as the "second" before time state for the next update cycle. Is this how other people go about it?