Facial action recognition (FAR) is an important but challenging task in computer vision due to the high variability of facial expression. To capture the complex facial features, diverse face representations need to be employed. In this paper, we propose a novel multikernel learning algorithm to effectively integrate heterogeneous features for FAR. We employ seven types of facial features, including Local Binary Patterns (LBP), Histograms of Oriented Gradients (HOG), convolutional neural network (CNN) features, Gabor Wavelets, and so on, and learn a combination of these different feature types in a unified way. Two single kernel support vector machine (SVM) models and a multiple kernel learning SVM model are compared in our experiments. By combining all seven feature types, we achieve an accuracy of 62.02% on the JAFFE dataset, which is far better than the performance of any single feature type. Moreover, our proposed multikernel learning method achieves a competitive accuracy.

Active appearance model (AAM), facial action unit (AU), facial expression recognition and analysis (FERA), local Gabor binary pattern (LGBP), multikernel learning.