2
$\begingroup$

Given a 2D image of a human face and the corresponding 3D shape, how do I project the shape onto the images to extract texture patches from them to form a partial UV map?

$\endgroup$

1 Answer 1

1
$\begingroup$

I've accomplished this by positioning the mesh in a known position (such as the origin), placing the image in front of it so that from the camera's point of view they align as desired. Then it's simply a matter of shooting a ray from each polygon's vertices through the image towards the camera. Wherever the ray intersects the image, that's your (u, v) point for that vertex.

$\endgroup$
5
  • $\begingroup$ have you used some algorithm to position the mesh and the place the image or have you done it manually? $\endgroup$
    – Abhinay B
    Commented Feb 20, 2018 at 4:38
  • $\begingroup$ In my cases, users of my program would do it manually. If you're dealing with faces, there are various libraries that will give you key points on the face, like eyes, mouth, etc. I would check OpenCV to see if there's something like that, for example. If you're on iOS, I believe they have a library that can give you such info. $\endgroup$ Commented Feb 20, 2018 at 5:48
  • $\begingroup$ I know how to find the keypoints on the image of the face but how do i do the same on the mesh? if i can get that as well, then its simply a matter of shooting a ray from a keypoint in the mesh to the corresponding key point in the image to obtain the UV. Any ideas? $\endgroup$
    – Abhinay B
    Commented Feb 20, 2018 at 10:12
  • $\begingroup$ I've not done it before, but I wonder if you could render the mesh with a harsh directional light coming from above, slightly in front, and to one side, so that the eye brows and nose cast distinct shadows, and then use the same method as used for the image? $\endgroup$ Commented Feb 20, 2018 at 16:49
  • $\begingroup$ Using shadows may not be a good idea as all the feature points in a face may not captured. For instance, the eyes, parts of mouth,etc. Instead, using the landmarks provided in [github.com/anilbas/BFMLandmarks ] for the Basel Face model can be used to find the correspondence. $\endgroup$
    – Abhinay B
    Commented Feb 22, 2018 at 3:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.