I'm trying to implement an Hopfield Network for pictures of 32x32 bits either 1 or -1;

I have these 3 pictures and I transform each of them in a vector of 1024 elements. Then I take the 3 vectors and I build a matrix M:

U = [v1 v2 v3]

So I compute:

M = (U*U')/1024 

where M is a matrix of 1024*1024 elements and I set the diagonal to 0. in order to get the correlation matrix scaled by 1024 (Hebbian rule).

Now, theoretically each vector v should be a fixpoint. So I compute M*v. I take the sign and I compare with v. The problem is that they are different, so v is not a fixpoint. What am I doing wrong?

  • $\begingroup$ Try working this out on paper (without computation). Should it work when you take only one vector? When you take two vectors? $\endgroup$ – Yuval Filmus Jul 15 '17 at 10:56
  • $\begingroup$ @YuvalFilmus I have tried this with some small example and it works. Are my examples just lucky? $\endgroup$ – f.saint Jul 15 '17 at 11:50
  • $\begingroup$ Try proving the general case, starting with one vector and moving to two and then three. See if it really works. $\endgroup$ – Yuval Filmus Jul 15 '17 at 12:00
  • $\begingroup$ @YuvalFilmus I can understand that the property doesn't hold in general because, otherwise, it would have worked in this case also. What I would like to know is: should I do some preprocessing to the vectors in order to make this property holding? Should I use smth else instead? Some paper that I can read about what could I use to make the vector a fixpoint? $\endgroup$ – f.saint Jul 15 '17 at 12:06
  • $\begingroup$ I'm guessing that most of the time, the vector are approximate fixed points. Textbooks or the original paper should help. $\endgroup$ – Yuval Filmus Jul 15 '17 at 12:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.