Both spiking neural networks created with the Neural Engineering Framework (NEF) and Recurrent Neural Networks (RNNs) can be connected recurrently to exhibit neural dynamics. What is the difference between the set of dynamics that they can approximate and/or exhibit?


There can be a lot of different ways in which recurrent neural networks can be used. RNNs can have any activation function (logistic sigmoid is most commonly used), and they can be multilayer. There are different algorithms which can be used to train them e.g., backpropagaion and different optimization techniques e.g., hessian free optimization which can be used to optimize the networks for a particular problem.

NEF has been used for simple recurrent connections (for implementing memory circuit), however, we need to do some work and try out multi-layer networks in order to truly figure out what the differences in dynamics would be.

We have also tried hopfield networks with pre-calculated weight matrices, and using sigmoid neurons and were able to achieve similar dynamics forming attractors. However, implementing hopfiled with dynamic learning of weight matrices is something which needs to be tried out.

I suspect that using sigmoid activation functions in NEF and applying the same learning algorithms should give results very similar to RNNs (there might be some differences caused by synaptic filtering). However, if one uses an LIF curve as an activation function, then there is a possibility of greater differences.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.