Faces are the most fascinating objects for human beings. We are never tired of looking at faces, a fact used heavily by advertising companies. In the course of our childhood, we develop a remarkable expertise at deciphering the most subtle aspects of a face, such as recognizing identity and sex, but also noticing, for example, signs of tiredness, sadness or age. I am currently investigating what kind of information we extract from faces either for recognizing them ("this is a picture of Marc") or categorizing them ("these are all Asian faces"). Furthermore, I am testing the importance of body size for face recognition (embodied cognition). In my research I use primarily face images derived from our face database, psychophysical methods, eye- tracking and immersive virtual environments (in collaboration with the PAVE group).

Together with Johannes Schultz, I lead the group Recognition and Categorization of the department Human Perception, Cognition and Action.

Projects in collaboration with PhD students of the Recognition and Categorisation group include:

• Janina Esins: Characterizing face and object recognition in face-blind people (prosopagnosics). Her work is supervised by Johannes Schultz and me, external collaborator is Ingo Kennerknecht.
• Katharina Dobs: Investigating the interplay between faces and idiosyncratic facial motion in identity recognition tasks. Her work is supervised by Johannes Schultz, and myself, together with Cris Curio and Mario Kleiner from the Cognitive Engineering group).

Teaching: 

• At the Graduate School for Neural and Behavioral Sciences, I give a seminar on object and face recognition together with Johannes Schultz.
• Since 1999, I am organizing the yearly Practical Course/Blockpraktikum "Cognitive and Computational Psychophysics" (4 SWS).
• With Heinrich H. Bülthoff, I have been participating in teaching a course on Perception, Cognition and Action: Fundamentals and Research Methodology at the Department of Brain and Cognitive Engineering, Korea University (South Korea) since 2009.

What gives a face its ethnicity?    We can quickly and easily judge faces in terms of their ethnicity.  In a series of studies, we investigate various aspects pertaining to ethnicity and the "other race effect". This work is done in part in collaboration with Korea University (BioCyb Lab in the Department of Brain and Cognitive Engineering) and involves participants of different cultural background and expertise in terms of face ethnicities. Furthermore, we used face stimuli derived from our database of Asian and Caucasian faces.  

Interplay between sex and identity recognition in familiar faces.   We are very good at recognizing familiar faces. In this project, I test the accuracy of our memory of familiar faces. Futhermore,  I am investigating the impact of idiosyncratic facial features on sex classification.

Face recognition: Size does not matter.   The concept of “Embodied Cognition” implies that our own bodies, the way we act with our bodies, and the way our bodies “fit” into the environment, should all have important implications for our mental representation of the world. Thus the question arises whether we represent and/or process faces in a different way depending on our body size. This work is done in collaboration with Ian Thornton (University of Swansea, UK) and Betty Tesch (Mohler). For more details on one aspect of this project, see the report below.

   **************************************************************** 

   ****************************************************************

Investigating face recognition of active observers using full-bodied avatars in a virtual environment

I. Bülthoff      

Introduction

Persons who are much taller or smaller than most others might develop different representations of the world around them and acquire expertise at processing other specific views of their environment. We had looked at the specific case of face recognition in a previous series of desktop experiments and there was no evidence of individuals’ height influencing their representation of others' faces or their ability to process specific views of faces [1]. However, in those experiments as in many others on face recognition [2], face orientation and body height were ambiguous as isolated faces were shown on a computer screen to a passive observer sitting on a chair.

Goals

We designed an experiment that (1) allowed to disambiguate height and orientation of the face stimuli used for face recognition and (2) presented 3D faces on a full body instead of isolated face images and (3) specifically examined the influence of learned viewpoints for face recognition when observers actively viewed 3D-faces.

Methods

A virtual museum was created that contains 20 full-bodied avatars (statues). Half of them were sitting; the others were standing (Figure 1a). Using a head-mounted display, observers walked through the museum three times, approached each statue and viewed them from any horizontal (yaw) angle without time restrictions. We equated eye-level – and thus simulated height -- for all participants and restricted their vertical movement to ensure that the faces of sitting avatars were always viewed from above and standing avatars from below with the same pitch (vertical angle). After familiarization, recognition was tested using a standard old-new paradigm in which 2D images of the learnt faces were shown from various viewpoints (Figure 1b).

Initial results

Figure 2 shows the average performance during the test phase for correctly classifying never seen faces as new and faces that had been viewed in the museum as old. The answers to old faces are separated in two groups. The groups old-congruent and old-incongruent correspond to faces viewed in the test phase either under the same orientation as during learning or under a different orientation, respectively. Participants were significantly better and faster at recognizing faces in the congruent than in the incongruent group (t(23)= 17.16, p< 0.001 and t(23)=-4.13, p = < 0.001 respectively).

Initial conclusion

We found a clear influence of learned viewpoint during familiarization. Faces of sitting avatars were recognized more quickly and accurately when viewed from above than from another orientation. Thus, recognition of newly learned faces appears to be view-dependent in terms of pitch angle. Our failure to find a height effect in our previous study suggests that the variety of views of human faces experienced during a lifetime and possibly the preponderance of conversational situations between humans at close range typically counteracts any influence that body size might have on a person’s viewing experience of others’ faces [3].

Figure 1

Left: partial view of the virtual museum. Right: Incongruent (top) and congruent (bottom) test views of the face of a sitting avatar in the partial view.

Figure 2

Left: Accuracy results in percent correct (ordinate) for the new (New) and old faces show in a congruent (Old-c) or incongruent (Old-ic) orientation. Right: Reaction times in ms (ordinate) for the same groups of faces. Error bars represent SEM.

References             

1. Bülthoff, I, Wolf ,W & Ian M. Thornton, I. M.  (2009). Does your height affect the way you represent faces? Journal of Vision 9  503.

2. Wallraven, C., Schwaninger, A., Schuhmacher, S., & Bülthoff, H.H. (2002). View-Based Recognition of Faces in Man and Machine: Re-visiting Inter-Extra-Ortho 2nd international Workshop on Biologically Motivated Computer Vision, Tübingen, Germany. Lectures Notes in Computer Science, 2525, 651-660.

3. Bülthoff, I, Shrimpton, S & Ian M. Thornton, I. M.  (2011). Using avatars to explore height/pitch effects when learning new faces.  11th Annual Meeting of the Vision Sciences Society (VSS 2011) 11 136.

1979 Licence ès Sciences naturelles (equivalent to MA in natural Sciences in the US), University of Lausanne, Switzerland.

1983 Ph.D in Zoology, University of Lausanne, Switzerland. Doctoral Dissertation accomplished at the Max-Planck institute for Biological Cybernetics, Tübingen, Germany.

Academic and Research Experience

1977-1978            Teaching assistant in Zoology, University of Lausanne, Switzerland

1979-1983            Doctoral work. Doctoral Dissertation: “Visual mutants  of Drosophila melanogaster, functional neuroanatomical mapping of nervous activity by 3H-Deoxyglucose method”. Max-Planck institute for Biological Cybernetics, Tübingen, Germany

1983-1885            Postdoctoral fellow, Max-Planck-Institut für biologische Kybernetik, Tübingen, Germany, funded by the Swiss Research Foundation

1986-1991            Child rearing period (2 children)

1991-1993            Research assistant, Neuroscience Department, (Prof. Barry Connors), Brown University, RI, USA       

Since 09/1993       Researcher at the Max-Planck-Institut für biologische Kybernetik, Tübingen, Germany

Since 01/2009       Project leader at the Max-Planck-Institut für biologische Kybernetik, Tübingen, Germany

Major Research Interests

Investigating the mechanisms underlying face recognition.  At present my focus is on the following themes:

• The interplay between gender and identity information in face recognition
• The impact of voice distinctiveness on face recognition
• The influence of context and task on face recognition
• Crosscultural differences in face and object recognition
• The role of idiosyncratic viewing history in face recognition