When I implemented the paper Dynamic Fusion published in 2015 CVPR, I came into the trouble with how to solve the energy function:

$$E(W_t, V, D_t, \mathcal{E}) = \text{Data}(W_t, V, D_t) + λ\text{Reg}(W_t, \mathcal{E})$$


  • Wt is a set of 4x4 transformation matrix at time t;
  • V is a set of 3D vertices from canonical model;
  • Dt is a depth map at time t;
  • ε is used to find Wt matrices;
  • Data is a point-to-plane error;
  • Reg is a smooth term which enforce the two adjacent transformation matrix W to be similar.

To simplify the question, I try to solve the Data term and throw away Reg term.
$$E(W_t, V, D_t, \mathcal{E}) = \text{Data}(W_t, V, D_t)$$

Here is the Data term:

\begin{align*} r &= \text{Data}(W_t, V, D_t)= \sum_u n_u^{\intercal}\cdot (v_u-vl) \\ where,\text{v}_u &= W_t×\text{v}(u) \\ \text{n}_u &= W_t×\text{n}(u) \end{align*}

All the information but $W_t$ is known, use Gauss-Newton to solve the parameter:
$$\begin{gather*} \underset{W_t}{\text{minimize}}\ r^2 \\ W_t^{(t+1)} = W_t^t - (J^\intercal J)^{-1}Jr \end{gather*}$$

It seems to me that I need to calculate the Jacobian matrix $J$, so I expand r in matrix form:
$where\ W_t=$ \begin{bmatrix} 1 & -\gamma & \beta & t_x \\ \gamma & 1 & -\alpha & t_y \\ -\beta & \alpha & 1 & t_z \\ 0 & 0 & 0 & 1 \end{bmatrix} $v, n, vl$ are all 4x1 vector. (sorry, I don't know how to display matrix multiplication)

If I print all of the calculation, there are about 50 values containing $(\alpha,\beta,\gamma,t_x,t_y,t_z)$ to sum up for one Data term. My question is, can I directly do partial differential to the parameters $(\alpha,\beta,\gamma,t_x,t_y,t_z)$ of a $W_t$ matrix to get Jacobian matrix $J$? It it correct to update $W_t$ by $W_t^{(t+1)} = W_t^t - (J^\intercal J)^{-1}Jr$?
I cannot believe this is a good approach because it is kind of messy, and maybe inefficient. Please give me some advice how to solve this energy function, and I am curious why researchers only give energy function without explicit deviation on how to solve the function. Because energy function deviation is a common knowledge? If so, which course teach about it? I really want to take a course. Thanks.


1 Answer 1


That is not an equation to be solved; it is an objective function to be minimized. They apparently describe how they optimize it in Section 3.3.3 of the paper. I suggest trying to understand that section.

  • $\begingroup$ Thanks for the reply. Agree, one of the obstacles is to understand proper nouns mentioned in this sector, such as normal equation, pre-conditioned conjugate gradient descent solve, sparse Cholesky factorization, etc. Maybe I will know the solution after grasping these knowledge. $\endgroup$ Mar 17, 2021 at 6:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.