I've been reading some papers on reinforcement learning.
$$\Delta w=\frac{\partial ln\ p_w}{\partial w}r$$
I often see expressions, similar to the above one, where the weights (denoted by $w$) are updated following the partial derivative of the policy function (denoted by $p_w$) with respect to its weights.
But why do we take the $\log$? What is its purpose?
We often take the logarithm because:
Maximizing $\log \Phi(x)$ is equivalent to maximizing $\Phi(x)$, so in maximum-likelihood problems, we can maximize the log of the likelihood instead of maximizing the likelihood directly and the result will be equivalent.
The logarithm converts multiplication to addition, and the derivative of a sum is "nicer" than the derivative of a product. Products often arise when maximizing likelihoods, because the probability of something can often be written as the product of other probabilities (e.g., when we are calculating the probability of that multiple independent events all happen). The derivative of the product is ugly, but the derivative of the log of the product is nice and simple.
Gradient descent often involves maximizing a likelihood, so we'll need to take derivatives. The expressions get simpler/cleaner if we can take the derivative of the log likelihood instead.
