Many everyday actions take place in a social context and affective information forms an important channel of social communication. The goal of EU-Project TANGO is to take these two familiar ideas one radical step further by focussing on the essential interactive nature of social communication, in the domain of non verbal communication based entirely on facial and bodily expression. We investigate interactions in real life contexts showing agents in daily life situations such as communication or navigation.

A central goal of the project is the development of an exact mathematical theory of emotional communicative behaviour. Based on such a theory combined with advanced methods from computer vision and computer graphics emotional interactions can be studied quantitatively in detail and can be transferred in technical systems that simulate believable emotional interactive behaviour. Current common motion synthesis techniques in computer graphics mainly focus on physical factors. The role of other factors, and specifically psychological variables, is neglected and not well understood. Based on the obtained experimental results and mathematical new generation of technical devices establishing emotional communication between humans and machines will be developed. TANGO goes beyond the sate of the art in theoretical scope, in methodological approaches and in innovative applications that are anticipated.
See also implications of understanding crowd behavior for driver assistance applications, workshop at Intelligent Vehicles Symposium, Baden-Baden.

Giese MA , Chiovetto E and Curio C (September-2012): Perceptual relevance of kinematic components of facial movements extracted by unsupervised learning, 35th European Conference on Visual Perception, Alghero, Italy, Perception, 41(ECVP Abstract Supplement) 150.

de la Rosa S , Miekes S , Bülthoff HH and Curio C (September-2012): View dependencies in the visual recognition of social interactions, 35th European Conference on Visual Perception, Alghero, Italy, Perception, 41(ECVP Abstract Supplement) 240.

We have developed a novel facial expression animation control system that realizes a real-time version of the animation pipeline developed in our institute [Curio et al, 2010, APGV 06/ 08]. The device produces no noticeable feedback latencies for subjects between executing dynamic expressions and the perception of themselves animated, in e.g. a mirror-setup. Expressions are encoded currently by means of 3D Facial Action Units. This novel real-time visual feedback offers a variety of on-line dynamic manipulations. For non-experienced programmers an intuitive application programming interface (API) based on the public ‘Visual Psychophysics Toolbox Version3’ offers implementation of experimental protocols for facial movement analysis and graphical feedback by control of gains of Action-Unit signals, the exchange and delay of Action-Unit-components, and allow highly synchronized monitoring of multimodal physiological recordings of e.g. facial EMG, EEG and sound.

 


Curio C , Kleiner M , Breidt M and Bülthoff HH (January-2010): The Virtual Face Mirror Project: Revealing Dynamic Self-Perception in Humans, 4th International Conference on Cognitive Systems (CogSys 2010), 4(137).
CiteID: 6241

The Facial Action Unit based animation system we have developed in our group and it has been perceptually validated against other approaches of animation in a user study (Curio et al 2006). Besides development and evaluation this animation technology has been and is still currently applied to study all kinds of research questions in our department.
For example, we found a novel high-level after-effect in the perception of dynamic facial expressions (APGV 2008).  The controllable animation provides us with a novel continuous facial expression morph space. In analogy to the study of Leopold et al (Nature Neuroscience, 2001, Prototype-referenced shape encoding revealed by high-level aftereffects) we defined facial anti-expressions (see image left) to detect expression specific adaptation aftereffects during perception. Like using a continuous face identity morph space (from Leopold et al, Nature Neuroscience, 2001), visual adaptation to anti-expressions biases the recognition of expressions opposite the morph axis, specifically towards the original expression.

The recognition of faces is a fundamental visual function with importance for social interaction and communication. Scientific interest in facial recognition has increased dramatically over the last decade. Researchers in such fields as psychology, neurophysiology, and functional imaging have published more than 10,000 studies on face processing. Almost all of these studies focus on the processing of static pictures of faces, however, with little attention paid to the recognition of dynamic faces, faces as they change over time—a topic in neuroscience that is also relevant for a variety of technical applications, including robotics, animation, and human-computer interfaces. This volume offers a state-of-the-art, interdisciplinary overview of recent work on dynamic faces from both biological and computational perspectives.
 
Dynamic Faces Book, MIT Press, 2010: The chapters cover a broad range of topics, including the psychophysics of dynamic face perception, results from electrophysiology and imaging, clinical deficits in patients with impairments of dynamic face processing, and computational models that provide insights about the brain mechanisms for the processing of dynamic faces. The book offers neuroscientists and biologists an essential reference for designing new experiments, and provides computer scientists with knowledge that will help them improve technical systems for the recognition, processing, synthesizing, and animating of dynamic faces.
Topics: Computer-Vision, Computer Graphics, Human-Computer Interfaces, Computational Modeling, Psychophysics, Physiology and Clinics