As we all know, Matrix Factorization is an effective method to do rating prediction jobs in recommender systems. Thanks to the work of Yahuda Koren. My question is why MF can do this job? What's the physical meaning behind it?

  • 3
    $\begingroup$ Matrix factorization is a sort of (slightly lossy) compression. A sufficiently good compression contains (at least implicitly) a certain type of understanding about the compressed information. $\endgroup$ – Thomas Klimpel Mar 14 '14 at 6:14
  • 2
    $\begingroup$ Why do you think there has to be a "physical meaning"? $\endgroup$ – Raphael Mar 14 '14 at 9:37

The idea in matrix factorization is to find the latent variables which connect the input and the output. Suppose that we are interested in a movie recommendation system, and that movies "live" on a one-dimensional axis, having romantic comedies in one end, and action movies in the other. Each "input" and "output" move can be rated in this scale (say $1$ to $-1$). Given the input movies, we can estimate a person's location $p$ on the axis (say a point in $[-1,1]$) by taking the inner product between the movies they saw and their "ratings" $r_i$, i.e. $p \approx \frac{1}{n} \sum_{i=1}^n r_i$, where $1,\ldots,n$ are the movies seen by the person. Given the location, we can estimate how much that person will like other movies: movie $t$ will be liked by an amount proportional to $pr_t$ (this is just a simplistic linear model).

In this example, the recommendation matrix can be decomposed as an outer product $A_{it} = r_i r_t$. More generally, the situation could depend on several latent variables, and this is the general situation described by matrix factorization. The number of latent variables is the "middle" dimension in the factorization. If you have found a good factorization with only a few variables, then you have a succinct explanation of your data that could have predictive power.

  • $\begingroup$ Could you explain a bit more, or redirect me to an elaborate explanation? I don't see where the inner product is happening - the person's location is simply the average of the (ratings of the) movies he has seen. Furthermore, people and movies lie on the same axis? I'm missing something here. $\endgroup$ – elexhobby Mar 14 '14 at 15:02
  • $\begingroup$ Have you read this: datajobs.com/data-science-repo/…? If there is anything specific you don't understand, you can ask about it here, or you can try to look for it in some of the references. $\endgroup$ – Yuval Filmus Mar 15 '14 at 2:55
  • $\begingroup$ @elexhobby: Basically, if you assume that there are a small number of "prototypical" users (i.e., every user is a linear combination of a small number of other users) and similar for items, the recommendation matrix is low rank. Then you can try to get back the singular vectors (which represent the users/items) from the observations. Once you have them, you can predict. The point of this area is figuring out how to do it. $\endgroup$ – Louis Mar 15 '14 at 10:44

a rough intuition (other than the basic data compression concept) is as follows. recommender systems that use matrix factorization methods eg SVD can be seen as a set of linear weights of natural features found via unsupervised means. a "feature" eg for a movie database is a large set of movies of varying weights. they dont necessarily have recognizable human meanings but could be given loose interpretations based on correlations with other variables associated with the subjects. such as "movies that women like, movies that men like, movies that kids like, movies that republicans like," etcetera. an individuals tastes are then represented approximately as a linear weighted combination of all these "factors". hence its very similar to statistics-based Factor Analysis where, roughly, causes & effects are decomposed into weighted "factors". some accounts of the Netflix contest refer to this concept.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.