I am interested in the human processing of dynamic faces and its role for identity. The faces we encounter everyday are typically in motion, thus facial motion is assumed to contribute at least to some extent to the processing of identity. However, there are still many open questions remaining to bridge the gap between the well-studied static faces and the less well-understood processing of dynamic faces.

During my PhD, I will investigate the role of facial motion and its interaction with facial form in the processing of identity by applying different methods of neurosciences, i.e. psychophysics, neuroimaging and computational modeling.

Investigating idiosyncratic facial dynamics with motion retargeting

Introduction

The faces we naturally encounter everyday typically move. Previous studies have shown that facial motion carries information about the identity of a person and might comprise part of the information humans use for identifying faces [1, 2]. However, very little is known about how identity information in facial motion is processed and how it interacts with facial form. 3D facial animation systems provide well-controlled stimuli to study dynamic face processing and the interaction of facial motion and facial form. Despite this high level of control, such stimuli often lack naturalness due to artificial facial dynamics (e.g. movies made by linear morphing between two static images).

Goals

The overall goal is to investigate the role of idiosyncratic facial dynamics and its interaction with facial form for identity processing, and to uncover the neural structures involved. To this end, we need stimuli that allow disentangling facial motion from facial form. Here, as a first step, we assessed whether our method permits to capture and display idiosyncratic facial motion accurately.

Methods

We use a system that decomposes facial motion-capture data into time courses of basic action shapes [3]. The resulting time courses of an arbitrary number of basic action shapes can then be retargeted onto any morphable face model that uses the same semantic structure. In an initial psychophysics study, we captured different facial expressions using this system. The original time courses and five approximations were retargeted onto a 3D avatar head using basic action shapes created in Poser. After watching a video sequence of the actor performing an expression, participants were asked to choose which animation was most similar to the target video.

Initial results

We found that participants correctly chose the original retargeted facial motion rather than any of the five approximations. Approximations based on time courses of action shapes that are closer to the original led to more errors.

Initial conclusion

Our initial findings highlight the sensitivity of human perception to idiosyncratic facial dynamics. The results support the use of our facial animation system as a tool to investigate human processing of dynamic faces. Our system allows to disentangle facial motion from facial form and to create relatively natural stimuli with a high level of control. This approach will permit the investigation of the role of idiosyncratic facial dynamics for identity processing.

References

1. Knappmeyer B, Thornton IM and Bülthoff HH (2003) The use of facial motion and facial form during the processing of identity, Vision Research, 43, 1921-1936.

2. Hill H and Johnston A (2001) Categorizing sex and identity from the biological motion of faces, Current Biology, 11, 880-885.

3. Curio C, Breidt M, Kleiner M, Vuong QC, Giese MA and Bülthoff HH (2006) Semantic 3D motion retargeting for facial animation, 3rd Symposium on Applied Perception in Graphics and Visualization (APGV '06), ACM Press, New York, NY, USA, 77-84.

Figure 1.a Original time course of the basic action shape “Smile” captured by an actor with five approximations (linear with 0/1/2 control points, spline with 1/2 control points). b Performance in the psychophysics study. Participants correctly identified the original facial motion rather than any of the five approximations. Approximations closer to the original led to more errors.

Education

since 10/10: PhD student at Max Planck Institute for Biological Cybernetics, Tübingen, Germany (Dept. Bülthoff: Human Perception, Cognition and Action)

10/02 - 06/08: Diploma Psychology, Philipps-University Marburg, Germany

10/02 - 09/07: Diploma Computer Science, Philipps-University Marburg, Germany

Research and Teaching Experience

05/11 - 08/11: Supervision of Kathryn Bonnen (Michigan State University) working on “Physical and perceptual analysis of the 3D face database” as an internship

10/04 – 03/08: Student Research Assistant at the Cognitive Psychophysiology Lab (Prof. Frank Rösler), Philipps-University of Marburg, Germany

03/06 – 09/06: Visiting Research Assistant at the Laboratory of Systems Neurodynamics (Prof. William B Levy), University of Virginia, USA

Work Experience

02/09 - 07/10: IT Consultant / Software Engineer at PRODYNA AG, Frankfurt, Germany

07/08 - 01/09: Freelancer / Software Engineer in London, GB