I would like to have a compact conceptual explanation that allows me to gain some feeling with the concepts. The following list uses $x_i, y_i$ as events of experiments $X, Y$.
- Self-information, denoted as $I(x_i)$
- Mutual information, denoted as $I(x_i; y_i)$
- Uncertainty (which is closely related to self-information and/or mutual information, I believe).
- Average mutual information, denoted as $I(X, Y)$.
- Entropy, denoted as $H(X)$, which is average self-information, I believe.
Next, there are a couple of "conditional ..." defined.
Can you sketch a compact overview of these concepts and how they relate, all intuitively?