I am getting properly stuck into reinforcement learning and I am currently reading the review paper by Kober et al. (2013).
And there is one constant feature that I cannot get my head around, but which is mentioned a lot, not just in this paper, but others too; namely the existence of gradients.
In section 2.2.2. they say:
The approach is very straightforward and even applicable to policies that are not differentiable.
w.r.t. to finite difference gradients.
What does it mean to say that the gradients exist and indeed, how do we know that they exist? When wouldn't they exist?