When updating the weights of a neural network using the backpropagation algorithm with a momentum term, should the learning rate be applied to the momentum term as well?
Most of the information I could find about using momentum have the equations looking something like this:
$W_{i}' = W_{i} - \alpha \Delta W_i + \mu \Delta W_{i-1}$
where $\alpha$ is the learning rate, and $\mu$ is the momentum term.
if the $\mu$ term is larger than the $\alpha$ term then in the next iteration the $\Delta W$ from the previous iteration will have a greater influence on the weight than the current one.
Is this the purpose of the momentum term? or should the equation look more like this?
$W_{i}' = W_{i} - \alpha( \Delta W_i + \mu \Delta W_{i-1})$
ie. scaling everything by the learning rate?