# Is the Jacobian matrix calculation process correctly on this energy function?

When I implemented the paper Dynamic Fusion published in 2015 CVPR, I came into the trouble with how to solve the energy function:

$$E(W_t, V, D_t, \mathcal{E}) = \text{Data}(W_t, V, D_t) + λ\text{Reg}(W_t, \mathcal{E})$$

Where:

• Wt is a set of 4x4 transformation matrix at time t;
• V is a set of 3D vertices from canonical model;
• Dt is a depth map at time t;
• ε is used to find Wt matrices;
• Data is a point-to-plane error;
• Reg is a smooth term which enforce the two adjacent transformation matrix W to be similar.

To simplify the question, I try to solve the Data term and throw away Reg term.
$$E(W_t, V, D_t, \mathcal{E}) = \text{Data}(W_t, V, D_t)$$

Here is the Data term:

\begin{align*} r &= \text{Data}(W_t, V, D_t)= \sum_u n_u^{\intercal}\cdot (v_u-vl) \\ where,\text{v}_u &= W_t×\text{v}(u) \\ \text{n}_u &= W_t×\text{n}(u) \end{align*}

All the information but $$W_t$$ is known, use Gauss-Newton to solve the parameter:
$$\begin{gather*} \underset{W_t}{\text{minimize}}\ r^2 \\ W_t^{(t+1)} = W_t^t - (J^\intercal J)^{-1}Jr \end{gather*}$$

It seems to me that I need to calculate the Jacobian matrix $$J$$, so I expand r in matrix form:
$$where\ W_t=$$ $$\begin{bmatrix} 1 & -\gamma & \beta & t_x \\ \gamma & 1 & -\alpha & t_y \\ -\beta & \alpha & 1 & t_z \\ 0 & 0 & 0 & 1 \end{bmatrix}$$ $$v, n, vl$$ are all 4x1 vector. (sorry, I don't know how to display matrix multiplication)

If I print all of the calculation, there are about 50 values containing $$(\alpha,\beta,\gamma,t_x,t_y,t_z)$$ to sum up for one Data term. My question is, can I directly do partial differential to the parameters $$(\alpha,\beta,\gamma,t_x,t_y,t_z)$$ of a $$W_t$$ matrix to get Jacobian matrix $$J$$? It it correct to update $$W_t$$ by $$W_t^{(t+1)} = W_t^t - (J^\intercal J)^{-1}Jr$$?
I cannot believe this is a good approach because it is kind of messy, and maybe inefficient. Please give me some advice how to solve this energy function, and I am curious why researchers only give energy function without explicit deviation on how to solve the function. Because energy function deviation is a common knowledge? If so, which course teach about it? I really want to take a course. Thanks.

## 1 Answer

That is not an equation to be solved; it is an objective function to be minimized. They apparently describe how they optimize it in Section 3.3.3 of the paper. I suggest trying to understand that section.

• Thanks for the reply. Agree, one of the obstacles is to understand proper nouns mentioned in this sector, such as normal equation, pre-conditioned conjugate gradient descent solve, sparse Cholesky factorization, etc. Maybe I will know the solution after grasping these knowledge. Mar 17 '21 at 6:31