Given a 2D image of a human face and the corresponding 3D shape, how do I project the shape onto the images to extract texture patches from them to form a partial UV map?


I've accomplished this by positioning the mesh in a known position (such as the origin), placing the image in front of it so that from the camera's point of view they align as desired. Then it's simply a matter of shooting a ray from each polygon's vertices through the image towards the camera. Wherever the ray intersects the image, that's your (u, v) point for that vertex.

  • $\begingroup$ have you used some algorithm to position the mesh and the place the image or have you done it manually? $\endgroup$ – Abhinay B Feb 20 '18 at 4:38
  • $\begingroup$ In my cases, users of my program would do it manually. If you're dealing with faces, there are various libraries that will give you key points on the face, like eyes, mouth, etc. I would check OpenCV to see if there's something like that, for example. If you're on iOS, I believe they have a library that can give you such info. $\endgroup$ – user1118321 Feb 20 '18 at 5:48
  • $\begingroup$ I know how to find the keypoints on the image of the face but how do i do the same on the mesh? if i can get that as well, then its simply a matter of shooting a ray from a keypoint in the mesh to the corresponding key point in the image to obtain the UV. Any ideas? $\endgroup$ – Abhinay B Feb 20 '18 at 10:12
  • $\begingroup$ I've not done it before, but I wonder if you could render the mesh with a harsh directional light coming from above, slightly in front, and to one side, so that the eye brows and nose cast distinct shadows, and then use the same method as used for the image? $\endgroup$ – user1118321 Feb 20 '18 at 16:49
  • $\begingroup$ Using shadows may not be a good idea as all the feature points in a face may not captured. For instance, the eyes, parts of mouth,etc. Instead, using the landmarks provided in [github.com/anilbas/BFMLandmarks ] for the Basel Face model can be used to find the correspondence. $\endgroup$ – Abhinay B Feb 22 '18 at 3:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.