Standard Hopfield networks exhibit stable patterns (states) which are attractors of a dynamic system. I wonder how to modify standard Hopfield networks such that they exhibit stable limit cycles as attractors. Since for binary neurons (to which I'd like to restrict my question) each attractor is a limit cycle of some length (because state space is finite), the question asks for "short" limit cycles (significantly shorter than the size of the state space, but longer than $1$).

Is there a simple standard recurrent neural network (of which - optimally - Hopfield networks would be a special case, e.g. for some parameter $\lambda \rightarrow 0$) that typically gives rise to short limit cycles?

How many of these can there be (compared to the number of neurons), and how big can their added up basins of attraction be (compared to the size of state space)?

Toy example:

enter image description here

This mini network has one stable state $(00)$ with basin of attraction $\{00,11\}$ and one limit cycle $(10,01)$ with basin of attraction $\{10,01\}$

  • $\begingroup$ If what you are asking is for an example of such a thing, it should be easy to find one. Pick any function $f:\mathbb{R} \to \mathbb{R}$ that has a short limit cycle, then find a neural network that approximates $f$. Since neural networks are universal function approximators, such a neural network must exist. If you're asking how many can there be, I don't think such a question is well-defined, since the weights are real numbers and thus the number is presumably infinite (in fact, uncountable, most likely). Or maybe I've misunderstood -- I know nothing about Hopfield networks. $\endgroup$ – D.W. Jan 29 '18 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.