برآورد شدت های عملکرد صورت بر داده های 2D و 3D

ABSTRACT Estimation of facial action intensities on 2D and 3D data

Automated measurement of facial actions has many potential applications for intelligent human-computer interfaces and in behavioral science. Extracted facial actions can be used, for instance, to infer the emotional state of a person, to detect whether the driver appears too tired, or even they can be used for generating performance-driven facial animations. Facial Action coding System (FACS) [2] is the most common facial action measurement methodology which involves 44 Action Units (AUs) related to facial muscle activations that can be visually discerned. Being composed of very extensive set of rules, FACS requires certified human coders and coding is a very time consuming process. These issues also motivates development of automatic coders to be used in facial behavior research.

Although there is already a substantial literature on automatic expression and action unit recognition [3], it still continues to be an active area of study due to the challenging nature of the problem. However, in contrast to AU detection, there is much less work in the literature on AU intensity estimation. FACS defines AU intensities on a five-point ordinal scale, i.e., from lowest A to strongest level E intensity. The measurement of intensities can be useful in behavior research and for improved FACS coding. For instance, if the expression is surprise, and if AU 5 – Upper Lid Raiser is available, then, it should only be at B level. Second, estimating strength of the AUs we yield more information about mental state and emotional involvement of a subject. Furthermore, AU intensity outputs can be a basis for studying AU dynamics.

Most of the the works on expression intensity have investigated relationships between classification decision scores and intensities. For instance, Bartlett et al. [1] investigated correlations between intensity levels and SVM classifier margins of their Gabor filter-based detectors. They reported moderate to high correlations for several AUs. One criticism of using classifier scores is that they do not incorporate intensity knowledge. Yang et al. [9] have used the output scores of RankBoost based expression classifiers to better deal with intensity variations. They train RankBoost classifiers with onset to apex ranked image sequences, in order to rank the image pairs according to their emotion intensities. They obtained better image pair ordering performance than the linear SVM-margin approach in et al. [1]. However, though related, correctly identifying ranking of image pairs in a sequence whose intensity increases monotonically is quite a different problem than estimating intensities directly from single images. Besides, some additional techniques should be figured out to convert ordering of image pairs into intensities, also in a way that we have consistent intensity measurements between sequences. Recently Mahoor et al. [5] studied measurement of AU 6 and AU 12 intensities over six subjects via person-specific AAMs. They approach to intensity estimation as a classification problem and apply six level SVM classifiers by one-against-one technique. For feature extraction they perform AU specific dimension reduction by applying regularized locality preserving indexing on appearance data, and use delta features (i.e., by neutral face feature subtraction).

The main novelty of our work is in the application of a nonlinear regression scheme for AU intensity estimation. A second contribution is the detailed investigation of 3D modality for improved AU intensity estimation and the comparison and eventual fusion of 3D with 2D modality data. Finally, we experiment with 25 AUs, a much bigger variety, by a factor of three, of AU types treated in previous works in the literature.