I have a question that asks me "What is the expected number of observations in a state?" with the note: $$\sum^{\infty}_{d=1}d a^{d=1} = \frac{1}{(a-1)^2}\text{ when } |a| < 1$$ Prior to that I have a Markov Model with 3 states and the transition probabilities as follows:

  • S1: $P(S1) = 0.4$, $P(S2|S1) = 0.3$, $P(S3|S1) = 0.3$
  • S2: $P(S2) = 0.6$, $P(S1|S2) = 0.2$, $P(S3|S2) = 0.2$
  • S3: $P(S3) = 0.8$, $P(S1|S3) = 0.1$, $P(S2|S3) = 0.1$

Any guidance on what the note means or what I am trying to work out would be appreciated.


1 Answer 1


The note probably refers to the calculation of the expected value of a random variable $T_{i}$ that describes the time "we stay" in the state $S_i$.

From how I understand the question, this is exactly what you are tasked to compute. Therefore I understood the question as: "given that you are in some state $S_i$, how many observations of $S_i$ do you expect before changing a state?"

  • $\begingroup$ So I imagine for $S1$ you would expect $\approx 3$, $S2 \approx 6$ and $S3 \approx 25$? $\endgroup$ Commented Feb 27, 2022 at 15:04
  • $\begingroup$ I think you are correct, assuming this really is the intention of the question $\endgroup$
    – nir shahar
    Commented Mar 2, 2022 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.