A lot of activation functions in neural networks (sigmoid, tanh, softmax) are monotonic, continuous and differentiable (except of may be a couple of points, where derivative does not exist).
I understand the reason for continuity and differentiability, but can not really understand a reason for monotonisity.