1
$\begingroup$

I'm reading this paper on latent representations with the information bottleneck https://arxiv.org/pdf/1804.06216.pdf and in section three, the authors write that the parametric formulation of the information bottleneck is

$\max\limits_{\phi,\theta} - I_\phi(t;x) + \lambda I_{\phi,\theta}(t;y).$

My question is not about the information bottleneck entirely, but how to understand the first term $\max\limits_{\phi,\theta}$ where $\phi$ and $\theta$ are parameters of a model. This notation doesn't make sense to me as I usually expect something to follow the $\max$. Does it just mean the largest value over all the parameters in the model?

$\endgroup$
1
$\begingroup$

The notation stands for the maximum value of $-I_{\phi}(t;x) + \lambda I_{\phi,\theta}(t;y)$, where the maximum is taken over $\phi,\theta$. The result will be a function of $t,\lambda,x,y$.

Often in such a notation, the possible values of the variables being maximized over is restricted, and sometimes this forms part of the notation (say, $\max_{0 \leq \phi,\theta \leq 1}$). In other times, the possible values are restricted, but this is not reflected in the notation, and should be gleaned from context.

$\endgroup$
1
  • $\begingroup$ Thank you so much! I feel so stupid now haha. I thought the max operator was somehow its own term alongside the mutual information terms. I'm more used to the max operator on sets, i.e. max {set} with curly braces, which was the main source of confusion. So thank you once again. $\endgroup$ – Dalop Dec 29 '20 at 1:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.