# AEP with a Twist!

We know by AEP that if random variables $$X_1,X_2,...$$ are i.i.d. drawn from $$P_X$$ then the probability of the vectors in the weak typical set $$A_{\epsilon}^n = \{\vec x \in \mathcal{X}^n: |\frac{-1}{n}\log P(\vec x) - H(P_X)| \leq \epsilon \}$$ tends to 1.
I am working on a slightly different problem. Random variables $$X_1,X_2,...$$ are independently drawn from $$P^1_X,P^2_X,..,P^n_X$$ respectively. Let use denote $$\tilde P_X = \frac{1}{n} \sum_{i=1}^n P^i_X$$ (ie, the average of the distributions). Is it true that the sum of the vectors contained in the set $$B_{\epsilon}^n$$ would have a very high probability? $$B_{\epsilon}^n = \{\vec x \in \mathcal{X}^n: |\frac{-1}{n}\log P(\vec x) - H(\tilde P_X)| \leq \epsilon \}$$

$$= \{\vec x \in \mathcal{X}^n: |\frac{-1}{n}\sum_{i=1}^n\log P^i_X(x_i) - H(\tilde P_X)| \leq \epsilon \}$$ I have no idea how to prove/disprove this. It seems like I can not use Weak law of large numbers.

Suppose that $$X_n = n$$ with probability $$1$$. Then $$H(\tilde{P}_X) = \log n$$, but the only vector with non-zero probability has $$\frac{1}{n} \log \frac{1}{P(\vec{x})} = 0$$.
• I think the same answer is more insightful if you if define $X_n = n \mod N$ to restrict the random variables to finite symbols. $H(\tilde P_X) \approx \log N$ in that case. Thanks for the answer. Sep 21 '20 at 13:55
• Hey, I think the statement hols if I tweak it to $\frac{-1}{n}\sum \log \tilde P_X (x_i)$ in my definition of $B_{\epsilon}^n$ Sep 23 '20 at 13:40