How to detect facial landmarks - using haar and other way?

Question

I have done some projects using haar cascade(Ex.- Face-detection,Eye inside face detection,shoe detection etc), I also have some knowledge about it. I have trained my own models learning from youtube, this site etc.

Now I want to take it to the next level.

I want to detect facial landmarks, I have used different modules to do this, I have used dLib. But I want to learn more about it and as far as I know using multiple haar will be very much inefficient.

So how can I do this?

What I want to detect from the image :

Answer 1

There is a recent survey paper on exactly this topic: Wu & Ji, Facial Landmark Detection: A Literature Survey, International Journal of Computer Vision 127:2, February 2019, pp 115-142. That should give you pointers to everything you might want to know.

If you don't have access to Springer, here's the copy on arXiv.

Answer 2

Alternatively you should consider using deep learning for this. Current state-of-the-art methods in recognition (including facial landmark detection) are based on neural networks.

Several notable papers you might want to read are "OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields" and "Simple Baselines for Human Pose Estimation and Tracking". In the given application people mostly predict human skeleton keypoints, however one can easily do it for facial landmarks too, just by changing the dataset, target and retraining.

Unfortunately dlib is not the best tool for deep learning, people mostly use keras (tensorflow) and pytorch.