A question about Laplace's approximation:

In Laplace's method, we need to find the mode of a function and take second order Taylor's expansion. The first order term will vanish (since the gradient is zero at local optimum), and the second order term will be used to give a covariance of the gaussian. I am wondering if I am doing MCMC and the "point" I found is not local optimum (a.k.a gradient is not zero), will that influence my covariance matrix? Will the matrix be non-positive-definite something like that ?

  • $\begingroup$ Could you clarify this question? It seems to be about numerical analysis, which some computer scientists (such as myself) aren't overly familiar with. In particular, can you expand some of your acronyms, and give a bit of background as to Laplace's approximation i.e. what it is, where it's used? $\endgroup$ – jmite Jul 19 '13 at 16:51
  • $\begingroup$ It's about machine learning and probability. Sometimes we need to estimate the posterior distribution of some models ( some a gaussian prior times a logistic function ), but the distribution is too complicated to write down a analytic form. Laplace's approximation approximate the posterior by putting a gaussian distribution at its mode. ece.rice.edu/~vc3/elec633/graphical_models_notes_091108.pdf $\endgroup$ – Jing Jul 20 '13 at 23:38
  • $\begingroup$ You're probably going to have more luck posting this on Cross-Validated, the SO for statistics. As @jmite points out, this question is light on the computer science and heavy on the statistics. $\endgroup$ – Richard D Jul 21 '13 at 12:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.