2
$\begingroup$

The topic of extracting the Facial Action Coding System (FACS) Action Units (AUs) [1] from images and it's translation into emotion prediction [2] is pretty well studied, but I'm not clear on how it stacks up against alternative approaches such as Convolutional Neural Networks (CNN). How much does the accuracy of AUs effect the accuracy of emotion detection?

  1. "Joint Facial Action Unit Detection and Feature Fusion: A Multi-conditional Learning Approach", Eleftheriadis S, Rudovic O, Pantic M. (2016)
  2. "Emotion Detection From Facial Expression And Its Use In The Evaluation Of Stress" Suvashis Das (2013)
|cite|improve this question
$\endgroup$
2
$\begingroup$

As a preliminary answer, I can't imagine the extraction of AUs affecting their ability to detect emotions, since according to Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition?, CNNs trained to extract emotions already correspond to AUs (called FAUs in the paper), as shown in the table below taken from their paper:

CNN filter to

Additionally, this is a very basic CNN with only three convolutional layers and grayscaled inputs. Given a larger CNN using Inception-layers and residual connections, more specialisation would probably be seen.

However, this conclusion also makes me feel like I'm dismissing a whole area of research, so it's probably a little hasty.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Have you considered the metric used in AU (the divergence)? The AU correspond to CNN, but they differ a lot, one missing point is that FACS discards a lot of data, while CNN uses that part, this makes it harder to compare. I do not have a full answer (I haven't read one of the articles yet), but let me hint that CNN is capable of registering subactivisation and near to that threshold indicators, while AU is superior to clean, high amplitude emotions reading. $\endgroup$ – Evil Dec 18 '16 at 19:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.