# Backprop formula question

I'm reading this chapter https://www.deeplearningbook.org/contents/mlp.html of the Deep Learning book, and on page 209, they have this equation (assume there is no regularizer and no bias parameter): $$\nabla_{W^{(k)}}J = gh^{(k-1)T}$$ I'm not sure how this equation is derived, because I thought based on the equation they give on page 203: $$\nabla_xz = \left(\frac{\partial y}{\partial x}\right)^T\nabla_y z$$, then should we have $$\nabla_{W^{(k)}}J = \left(\frac{\partial a^{(k)}}{\partial W^{(k)}}\right)^T\nabla_{a^{(k)}} J = h^{(k-1)T}g$$ (because $$g=\nabla_{a^{(k)}}$$ and $$a^{(k)} = W^{(k)}h^{k-1}$$) instead?

• Check the dimensions of the two formulas – you're giving a scalar, whereas the book is giving a matrix. – Yuval Filmus Apr 4 '20 at 7:27