The Hessian of multi-layered network exhibits known behaviour at critical points as shown in . The tools of random matrix theory allow  to deduce the asymptotic distribution of the eigenvalues of the Hessian.
There are some numerical experiments with distributed Newton type stochastic methods. The scalability (ability to perform computations in parallel architecture ) is achieved under certain assumptions on the eigenvalues. I am just wondering about two questions:
1) Can one describe the eigenvalues distribution for the Hessian obtained from average reward function in reinforcement learning?
2) If yes, are there any scalable iterative methods (apart from SGD) that can optimise the average reward function and exploit the structure of the eigenvalues of the hessian if possible.
I beg my apology for possible misleading formulations. Please feel free to express your comments/concerns to me and I will edit the post accordingly.
 L. Sagun, L. Bottou, and Y. LeCun. Eigenvalues of the hessian in deep learning: Singularity and beyond. arXiv preprint arXiv:1611.07476, 2016.
 J. Pennington and Y. Bahri. Geometry of neural network loss surfaces via random matrix theory. In International Conference on Machine Learning, pages 2798–2806, 2017.