How to interpret plot of cost function in gradient descend algorithm?

I have a data set to which I want to apply logistic regression. I set my cost function to be

$J(\theta) = -\frac{1}{m} [\sum\limits_{i=1}^{m} y^{(i)} log{(h_\theta \cdot x^{(i)})} + (1-y^{(i)}) log(1-h_\theta(x^{(i)}))$

and I am running gradient descend algorithm.

When I plot the cost function as a function of the number of iterations, I am getting a plot like this Which I think is a good sign.

However, when I zoom in on the initial portion of the curve, I get this plot I don't know what to make out of the rising and falling in the cost in the beginning. Is that something that we can expect? If so, why? Shouldn't the gradient descend algorithm decrease the cost after every step? How is it that after some iterations there is an increase in the cost? I know that it is possible for gradient decent not to converge if the learning rate is big enough but the algorithm does seem to converge eventually. So why is it that there are some instances where gradient descend tends to increase the cost?

• There might be some numerical issues. I wouldn't worry too much. Also, gradient descent is only guaranteed to decrease the function if you take infinitesimal steps (or if the objective function is linear). Apr 22 '17 at 15:26
• Please take a look at gradient descent optimization. Yes, the fluctuations are expected, otherwise the step is too small and the number of iterations greatly increases (or the data is somewhat special).
– Evil
Apr 22 '17 at 16:23 