Why is channel capacity of AWGN infinite?

Question

My professor taught us that channel capacity of AWGN channel is infinite without any input power constraints. The noise is $Z \sim \mathcal{N}(0,\sigma^2) $. There is no constraint on input signal. I don't understand how the professor can directly say the channel capacity is infinite. Don't we need to maximize mutual information between input and output to get the channel capacity? How to do that for continuous variables?

Answer 1

Here is a coding scheme that demonstrates the main idea:

Encoding: Let the power constraint $P$ be fixed and suppose we want to transmit one information bit a time. We set our coding scheme as $X(0) = \sqrt{P}, X(1) = -\sqrt{P}$, where $X$ is the encoding function.

Decoding: Let $Y$ denote the received signal and $Z$ the additive Gaussian noise like you defined. We set the decoder as $\hat{X} = \mathbb{1}_{\{Y > 0\}}(Y)$, where $\mathbb{1}_A(w)$ is the indicator function that yields $1$ if $w \in A$ and $0$ otherwise.

Probability of error: Let $P_e$ denote the probability of error. We assume the information bits are equally likely, since otherwise we could simply use optimal source coding to ensure they are. Then,

\begin{align} P_e &= \frac{1}{2}P(Y>0 | X = -\sqrt{P}) + \frac{1}{2}P(Y \leq 0 | X = \sqrt{P}) \\ &= \frac{1}{2}P(Z > \sqrt{P} | X = -\sqrt{P}) + \frac{1}{2}P( Z \leq -\sqrt{P} | X = \sqrt{P}) \\ &= P(Z > \sqrt{P}) = 1 - \Phi\left(\sqrt{\frac{P}{\sigma^2}}\right), \end{align}

where $\Phi(t) = \int_{-\infty}^t \frac{1}{\sqrt{2\pi}}e^{-\frac{-t^2}{2}}$ is the Gaussian cdf. The key observation here is that, as a cdf, this is a non-decreasing function that converges to $1$ in the limit. By increasing $P$, we can make it arbitrarily close to $1$. In other words, let $\epsilon > 0$, for large enough $P$, $P_e < \epsilon$. Without a power constraint, we can send one bit of information with arbitrarily small probability of error. This coding scheme proves a rate of $1$ is achievable.

Ok, so how do we get from an achievable rate of $1$ to $\infty$? Let's see what happens when we increase our rate from $1$ to $2$, by encoding two information bits at a time. Let $$X(b_1, b_2)=\begin{cases} \sqrt{P}, &\text{ if } (b_1,b_2) &= (0,0) \\ \frac{\sqrt{P}}{2}, &\text{ if } (b_1,b_2) &= (0,1) \\ -\sqrt{P}, &\text{ if } (b_1,b_2) &= (1,0) \\ -\frac{\sqrt{P}}{2}, &\text{ if } (b_1,b_2) &= (1,1) \end{cases}$$

Now, if you follow the same procedure as above, you'll find out that $P_e = P\left(Z > \frac{\sqrt{P}}{2} \right) = 1 - \Phi\left(\sqrt{\frac{P}{4\sigma^2}} \right)$. Therefore, we can again find a (larger) $P$ that allows us to squeeze $2$ information bits into $1$ coded bit with an arbitrarily small probability error. As you can imagine, if $P$ is unbounded, we can simply keep doing this to encode more and more information bits to in a single coded bit without sacrificing from $P_e$.

Moral of the story: Without a bound on transmission power, we can pick a set of coded bits (codewords of length 1) that achieve arbitrarily small $P_e$ and we can do this for an arbitrarily large set of code bits to squeeze as many information bits into 1 as we want. So, the achievable rates is unbounded and since the capacity is the smallest upper bound on the set of achievable rates, it is $\infty$.