This is a question about Markov Models. Let's say we have the following situation enter image description here

Let's say that we want to find the probability that $2$ rainy days follow a nice day. You'd simply have $0.25 \cdot 0.5=0.125=12.5\%$. However let's take it up a notch. What's the probability that on the $7^{th}$ day it's snowy? Or in general how would you find the probability that on the $n^{th}$ day it's a certain weather?

I think that you could take one possibility of a sequence of events, such that on the $7^{th}$ day it snows, such as nice, rain, snow, nice, rain, snow and snow which has a probability of $0.1953\%$ and then add all such probabilities but I'm not a 100% sure.

  • 1
    $\begingroup$ I'm confused. The fact that it's impossible to have days of nice weather in a row suggests that you live in the UK, but we don't say "ski mask" here. :-) $\endgroup$ – David Richerby Apr 10 '18 at 15:41

Yes, you just add up the probabilities of all the possibilities. However, there's a conceptually clearer way of doing it that's less error-prone.

The transition matrix tells you the probability of moving from one state to another in one step. The probability that it rains the day after tomorrow given that it's nice today is, as you've seen, given by $$\Pr(N\to N)\,\Pr(N\to R) + \Pr(N\to R)\,\Pr(R\to R) + \Pr(N\to S)\,\Pr(S\to R)\,.$$ If you look at this closely, you'll see that it's exactly the (Nice, Rain) entry of the square of the transition matrix. Similarly, the $n$th power of the transition matrix tells you the probability of moving between two states in $n$ steps.

Powers of the transition matrix can be computed efficiently with repeated squaring – for example $M^7 = (M^2)^2\times M^2\times M$ and there's the bonus that you get all the $n$-step probabilities instead of just one.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.