GieseCC2012 7 MA Giese E Chiovetto C Curio Alghero, Italy2012-09-00 150 35th European Conference on Visual Perception The idea that complex facial or body movements are composed of simpler components (usually referred to as 'movement primitives'or 'action units') is common in motor control (Chiovetto 2011 Journal of Neurophysiology105(4), 1429-31.) as well as in the study of facial expressions (Ekman and Friesen, 1978). However, such components have rarely been extracted from real facial movement data. Methods: Combining a novel algorithm for anechoic demixing derived from (Omlor and Giese 2011 Journal of Machine Learning Research121111-1148) with a motion retargetting system for 3D facial animation (Curio et al, 2010, MIT Press, 47-65), we estimated spatially and temporally localized components that capture the major part of the variance of dynamic facial expressions. The estimated components were used to generate stimuli for a psychophysical experiment assessing classification rates and emotional expressiveness ratings for stimuli containing combinations of the extracted components. Results: We investigated how the information carried by the different extracted dynamic facial movement components is integrated in facial expression perception. In addition, we tried to apply different cue fusion models to account quantitatively for the obtained experimental results. no notspecified http://www.kyb.tuebingen.mpg.de/ published -150 15017 15422 7016 7 M Breidt HH Bülthoff C Curio Santa Barbara, CA, USA2011-03-00 713 719 Ninth IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011) Rich face models already have a large impact on the fields of computer vision, perception research, as well as computer graphics and animation. Attributes such as descriptiveness, semantics, and intuitive control are desirable properties but hard to achieve. Towards the goal of building such high-quality face models, we present a 3D model-based analysis-by-synthesis approach that is able to parameterize 3D facial surfaces, and that can estimate the state of semantically meaningful components, even from noisy depth data such as that produced by Time-of-Flight (ToF) cameras or devices such as Microsoft Kinect. At the core, we present a specialized 3D morphable model (3DMM) for facial expression analysis and synthesis. In contrast to many other models, our model is derived from a large corpus of localized facial deformations that were recorded as 3D scans from multiple identities. This allows us to analyze unstructured dynamic 3D scan data using a modified Iterative Closest Point model fitting process, followed by a constrained Action Unit model regression, resulting in semantically meaningful facial deformation time courses. We demonstrate the generative capabilities of our 3DMMs for facial surface reconstruction on high and low quality surface data from a ToF camera. The analysis of simultaneous recordings of facial motion using passive stereo and noisy Time-of-Flight camera shows good agreement of the recovered facial semantics. no notspecified http://www.kyb.tuebingen.mpg.de/ published 6 15017 15422 6996 7 M Breidt HH Bülthoff C Curio Seoul, Korea2010-12-00 1 2 3rd ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia 2010) Affordable 3D vision is just about to enter the mass market for consumer products such as video game consoles or TV sets. Having depth information in this context is beneficial for segmentation as well as gaining robustness against illumination effects, both of which are hard problems when dealing with color camera data in typical living room situations. Several techniques compute 3D (or rather 2.5D) depth information from camera data such as realtime stereo, time-of-flight (TOF), or real-time structured light, but all produce noisy depth data at fairly low resolutions. Not surprisingly, most applications are currently limited to basic gesture recognition using the full body. In particular, TOF cameras are a relatively new and promising technology for compact, simple and fast 2.5D depth measurements. Due to the measurement principle of measuring the flight time of infrared light as it bounces off the subject, these devices have comparatively low image resolution (176 x 144 ... 320 x 240 pixels) with a high le vel of noise present in the raw data. no notspecified http://www.kyb.tuebingen.mpg.de/ published 1 15017 15422 5961 2 C Walder M Breidt HH Bülthoff B Schölkopf C Curio MIT Press Cambridge, MA, USA 2010-12-00 255 276 Dynamic Faces: Insights from Experiments and Computation no notspecified http://www.kyb.tuebingen.mpg.de/ published 21 15017 1542215017 15420 6798 7 M Breidt HH Bülthoff C Curio Edinburgh, UK2010-10-00 11 ACM/SSPNET 2nd International Symposium on Facial Analysis and Animation (FAA 2010) Morphable models have proven to be very successful for analyzing and synthesizing 2D and 3D recordings of faces. They are used extensively in computer vision, computer graphics as well as in psychology research. The growing interest in extending this work from static to dynamic faces [Curio et al. 2010] has led us to build a large database of three-dimensional facial deformation data. In comparison to other databases, this database contains a large corpus of facial deformations that were carefully put into dense correspondence. From that we can obtain generative facial expression models to build a "4D Morphable Face Model", extending previous work for identity [Blanz and Vetter 1999]. no notspecified http://www.kyb.tuebingen.mpg.de/ published -11 15017 15422 5899 7 C Walder M Breidt HH Bülthoff B Schölkopf C Curio Jena, Germany2009-09-00 41 50 31. Symposium of the German Association for Pattern Recognition (DAGM 2009) We present a novel algorithm for the markerless tracking of deforming surfaces such as faces. We acquire a sequence of 3D scans along with color images at 40Hz. The data is then represented by implicit surface and color functions, using a novel partition-of-unity type method of efficiently combining local regressors using nearest neighbor searches. Both these functions act on the 4D space of 3D plus time, and use temporal information to handle the noise in individual scans. After interactive registration of a template mesh to the first frame, it is then automatically deformed to track the scanned surface, using the variation of both shape and color as features in a dynamic energy minimization problem. Our prototype system yields high-quality animated 3D models in correspondence, at a rate of approximately twenty seconds per timestep. Tracking results for faces and other objects are presented. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/WalderEtAl_DAGM2009_5899[0].pdf published 9 15017 1542215017 15420 3992 7 C Curio M Breidt M Kleiner QC Vuong MA Giese HH Bülthoff Boston, MA, USA2006-07-00 77 84 3rd Symposium on Applied Perception in Graphics and Visualization (APGV 2006) We present a system for realistic facial animation that decomposes facial Motion Capture data into semantically meaningful motion channels based on the Facial Action Coding System. A captured performance is retargeted onto a morphable 3D face model based on a semantically corresponding set of 3D scans. The resulting facial animation reveals a high level of realism by combining the high spatial resolution of a 3D scanner with the high temporal accuracy of motion capture data that accounts for subtle facial movements with sparse measurements. Such an animation system allows us to systematically investigate human perception of moving faces. It offers control over many aspects of the appearance of a dynamic face, while utilizing as much measured data as possible to avoid artistic biases. Using our animation system, we report results of an experiment that investigates the perceived naturalness of facial motion in a preference task. For expressions with small amounts of headmotion, we find a benefit for our part-based generative animation system that is capable of local animation over an example-based approach that deforms the whole face at once. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/apgv06-77_3992[0].pdf published 7 15017 15422 2889 7 C Curio M Giese Breckenridge, CO, USA2005-01-00 261 268 IEEE Computer Society Workshop on Motion and Vision Computing (MOTION 2005) Many existing systems for human body tracking are based on dynamic model-based tracking that is driven by local image features. Alternatively, within a view-based approach, tracking of humans can be accomplished by the learning-based recognition of characteristic body postures which define the spatial positions of interesting points on the human body. Recognition of body postures can be based on simple image descriptors, like the moments of body silhouettes. We present a system that combines these two approaches within a common closed-loop architecture. Central characteristics of our system are: (1) Mapping of image features into a posture space with reduced dimensionality by learning one-to-many mappings from training data by a set of parallel SVM regressions. (2) Selection of the relevant regression hypotheses by a competitive particle filter that is defined over a low-dimensional hidden state space. (3) The recognized postures are used as priors to initialize and support classical model-based tracking using a flexible articulated 2D model that is driven by local image features using a vector field approach. We present pose tracking and reconstruction results based on a combination of view-based and model-based tracking. Increased robustness and improved generalization properties are achieved even for small amounts of training data. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/CurioGiese05_2889[0].pdf published 7 2319 10 M Breidt C Wallraven DW Cunningham HH Bülthoff