I found an article on Catalytic space which describes how additional memory (which must be returned to it's arbitrary, initial state) can be useful for computation. There's also an expository follow up with some more details.
In particular, they describe a scheme for iterated matrix multiplication (for the purposes of this post, multiplying $n$, $n \times n$ matrices) in log space, poly "catalytic space", and polynomial time. The argument to the best of my understanding can be sketched as follows.
Theorem 4 (second article) says any arithmetic formula (i.e. arithmetic circuit w/ fanout 1) of depth $d$ can be computed with a program of size $4^d$ (and all the previously mentioned space guarantees). Here, "program" is in the context of register machines, and the size is the number of instructions and equals the runtime.
Brent et al. 1973 proved that any arithmetic formula of size $s$ can be "balanced" to have depth $O(\log s)$, so combining with (1) it has a program of size $poly(s)$
For some reason, I cannot find this last, implied claim in either of the articles: there is a arithmetic formula of size $s = poly(n)$ for iterated matrix multiplication. This would imply the claim made by the papers -- that IMM can be done in polynomial time with the other space bounds, but for some reason I can't find the claim explicitly written out, which suggests I am missing something.
The smallest formula I can think of for iterated matrix multiplication is "divide and conquer" on the number of matrices, and results in size $n^{O(\log n)}$, and I don't see any way to improve on this.
The first linked article says "iterated matrix product can be computed transparently by polynomial size programs", which would seem to follow by putting together 1,2, and 3, but it references an old thesis I can't find anywhere.
So, it's either the case that I've totally misread the argument, or there should exist a polynomial sized arithmetic formula for iterated matrix multiplication. Does anyone know of one?