# Observable Markov Model: Expected number of observations

I have a question that asks me "What is the expected number of observations in a state?" with the note: $$\sum^{\infty}_{d=1}d a^{d=1} = \frac{1}{(a-1)^2}\text{ when } |a| < 1$$ Prior to that I have a Markov Model with 3 states and the transition probabilities as follows:

• S1: $$P(S1) = 0.4$$, $$P(S2|S1) = 0.3$$, $$P(S3|S1) = 0.3$$
• S2: $$P(S2) = 0.6$$, $$P(S1|S2) = 0.2$$, $$P(S3|S2) = 0.2$$
• S3: $$P(S3) = 0.8$$, $$P(S1|S3) = 0.1$$, $$P(S2|S3) = 0.1$$

Any guidance on what the note means or what I am trying to work out would be appreciated.

The note probably refers to the calculation of the expected value of a random variable $$T_{i}$$ that describes the time "we stay" in the state $$S_i$$.
From how I understand the question, this is exactly what you are tasked to compute. Therefore I understood the question as: "given that you are in some state $$S_i$$, how many observations of $$S_i$$ do you expect before changing a state?"
• So I imagine for $S1$ you would expect $\approx 3$, $S2 \approx 6$ and $S3 \approx 25$? Commented Feb 27, 2022 at 15:04