Is it proper to view conditional probabilities, such as the forms:



P(a, b|c, d)

...and so forth, as being tensors?

If so, does anyone know of a decent introductory text (online tutorial, workshop paper, book, etc) which develops tensors in that sense for computer scientists/machine learning practitioners?

I have found a number of papers, but those written at an introductory level are written for physicists, and those written for computer scientists are rather advanced.

  • 1
    $\begingroup$ sorry but what is Tensors? $\endgroup$ – seteropere May 17 '13 at 23:31
  • 1
    $\begingroup$ Tensors are the multidimensional analogs of matrices. Matrices are two-dimensional tensors. $\endgroup$ – Yuval Filmus May 18 '13 at 5:35
  • $\begingroup$ I would have said they are the multidimensional analogs of vectors, myself. But, in that light, they are geometric objects with special rules and operators that I'm not deeply familiar with. So the question is, can conditional probabilities be viewed as tensors in that sense, and if so how are the typical operators interpreted? $\endgroup$ – Novak May 18 '13 at 15:20
  • 1
    $\begingroup$ If you view $P(A=a) = \sum_b P(A=a|B=b) P(B=b)$ as a matrix times a vector, then you can find similar interpretations of other similar equations. Is that what you're after? $\endgroup$ – Yuval Filmus May 21 '13 at 23:46
  • $\begingroup$ Yes, basically. With the host of questions that follow: Are those proper tensor operations? Is there an intuitive interpretation of covariance and contravariance in this framework? What about contraction operations? Raised and lowered indices, and so-called "index juggling"? $\endgroup$ – Novak May 22 '13 at 17:35

This is definitely possible, although the tensor has of course certain additional structure (constraints).

If you consider the following conditional defined for a categorical response $Y$ on categorical predictors $X_i$:


this correspond to a conditional probability tensor of size $d_0 \cdot d_1 \cdot \ldots \cdot d_n$. Here $Y$ (as a categorical response) takes $d_0$ levels.

Foremost the elements of the tensor have each to be nonnegative.

You can find papers on this, e.g. "Bayesian Conditional Tensor Factorizations for High-Dimensional Classification" by Yang and Dunson (pdf, arxiv).

Of course, if the random variables concerned are not categorical variables, but either continuous random variables or taking an infinite number of values (or both), the mapping to finite tensors won't be so trivial. You'll need something like an infinite tensor (as an extension of an infinite matrix).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.