Skip to main content
17 votes
Accepted

What is meant by the term "prior" in machine learning

Put simply, and without any mathematical symbols, prior means initial beliefs about an event in terms of probability distribution. You then set up an experiment and get some data, and then "update" ...
fade2black's user avatar
  • 9,837
3 votes

Solomonoff's theory of induction, Kolmogorov complexity and Bayesian Inference

Mathematics and computer science doesn't have anything to say about whether simpler hypotheses are more likely. That's a question about reality, not about math / computer science. What computer ...
D.W.'s user avatar
  • 162k
3 votes

Bayes theorem probability doesn't make sense

Well, then your data is clearly wrong!! Bayes Theorem or not, you simply cannot have $$P\left(B\,|\,A \right) \times P\left(A\right) > P\left(B\right) $$ but this "impossibility" is exactly what ...
ManRow's user avatar
  • 141
2 votes
Accepted

Is there any example of Regression Tree driven optimization (or active learning)?

An active learning approach using which combines an incrementally-learned Regression Tree with bandit-style sampling from leaf nodes to determine which instance to request a label for next is ...
Dennis Soemers's user avatar
2 votes

Bayes theorem probability doesn't make sense

The problem is your data! For example, the last row shows that $\mathbb{P}(A) = 1$ and $\mathbb{P}(B|A) = 0.8$. If $\mathbb{P}(A) = 1$ means $A$ is equivalent to all possiblities of event world. Hence,...
OmG's user avatar
  • 3,572
2 votes

What is meant by the term "prior" in machine learning

It's roughly any pre-training choices you encode into your model In machine learning a prior is, according to the book "Deep Learning" by Goodfellow, Bengio, and Courville, a probability ...
bob's user avatar
  • 121
1 vote
Accepted

What is the time complexity of the SGVB estimator?

Asymptotic running time analysis is not terribly useful for gradient descent used to train machine learning models. In practical machine learning, we run gradient descent for some fixed number of ...
D.W.'s user avatar
  • 162k
1 vote
Accepted

Density of uniform distribution over two disjoint squares

The density $p_R$ of the uniform distribution over a rectangle $R$ is given by $p_R(x) = 0$ if $x \notin R$, and $p_R(x) = 1/\mathit{area}(R)$ otherwise. Indeed, up to scaling the distribution must ...
Yuval Filmus's user avatar
1 vote
Accepted

Gaussian distribution with condition?

$$\mathcal{N}(x|m,\Sigma)=(2\pi)^{-k/2}|\Sigma|^{-1/2}\exp\left(-2^{-1}(x-m)^T\Sigma^{-1}(x-m)\right)$$ where $x,m$ are vectors of dimension $k$, $\Sigma$ is a $k\times k$ matrix, $|\Sigma|$ is its ...
plop's user avatar
  • 549
1 vote

Bayes theorem probability doesn't make sense

I agree with @OmG that the table is wrong. However, $\mathbb{P}(B\mid A)$ must be equal to $\mathbb{P}(B)$ and not $1$. Intuitively: as @OmG said, $\mathbb{P}(A)=1$ means that under $\mathbb{P}$ the ...
Jim's user avatar
  • 113
1 vote

What is meant by the term "prior" in machine learning

In Bayesian statistics, a "prior" represents the beliefs we have before observing some data. Then, after we observe some data, we update our beliefs; those updated beliefs are called the &...
D.W.'s user avatar
  • 162k
1 vote

Convergence of Markov model

This isn't a hidden Markov model; this is an ordinary Markov model. Take a look at Wikipedia's article on Markov chains and specifically the notion of a steady-state distribution (or stationary ...
D.W.'s user avatar
  • 162k
1 vote

Are the Confabulation Theories of Thaler and Hecht-Nielsen Isomorphic?

I don't think they map onto each other, aside from RHN's brief discussion of thalamo-cortical loops, the inspiration for SLT's so-called "Creativity Machines." In SLT's architectures, compelling/...
Pat's user avatar
  • 11
1 vote

What ML methods exist to categorize signal from noise? Red noise? Spatially correlated noise?

The basic approach is to model the noise as an appropriate stochastic process (aka random process). There's an entire subfield of statistics that has studied different kinds of stochastic processes, ...
D.W.'s user avatar
  • 162k

Only top scored, non community-wiki answers of a minimum length are eligible