CunninghamIS2011 28 D Cunningham T Isenberg SN Spencer 5753 28 DW Cunningham V Interrante P Brown J McCormack 5755 28 DW Cunningham GW Meyer L Neumann A Dunning R Paricio CastilloWC2014 3 S Castillo C Wallravem DW Cunningham 2014-05-00 3-4 25 225–233 Computer Animation and Virtual Worlds We can learn a lot about someone by watching their facial expressions and body language. Harnessing these aspects of non-verbal communication can lend artificial communication agents greater depth and realism but requires a sound understanding of the relationship between cognition and expressive behaviour. Here, we extend traditional word-based methodology to use actual videos and then extract the semantic/cognitive space of facial expressions. We find that depending on the specific expressions used, either a four-dimensional or a two-dimensional space is needed to describe the variance in the stimuli. The shape and structure of the 4D and 2D spaces are related to each other and very stable to methodological changes. The results show that there is considerable variance between how different people express the same emotion. The recovered space can well capture the full range of facial communication and is very suitable for semantic-driven facial animation. no notspecified http://www.kyb.tuebingen.mpg.de/ published -225 The semantic space for facial communication KaulardCBW2012 3 K Kaulard DW Cunningham HH Bülthoff C Wallraven 2012-03-00 3 7 1 18 PLoS One The ability to communicate is one of the core aspects of human life. For this, we use not only verbal but also nonverbal signals of remarkable complexity. Among the latter, facial expressions belong to the most important information channels. Despite the large variety of facial expressions we use in daily life, research on facial expressions has so far mostly focused on the emotional aspect. Consequently, most databases of facial expressions available to the research community also include only emotional expressions, neglecting the largely unexplored aspect of conversational expressions. To fill this gap, we present the MPI facial expression database, which contains a large variety of natural emotional and conversational expressions. The database contains 55 different facial expressions performed by 19 German participants. Expressions were elicited with the help of a method-acting protocol, which guarantees both well-defined and natural facial expressions. The method-acting protocol was based on every-day scenarios, which are used to define the necessary context information for each expression. All facial expressions are available in three repetitions, in two intensities, as well as from three different camera angles. A detailed frame annotation is provided, from which a dynamic and a static version of the database have been created. In addition to describing the database in detail, we also present the results of an experiment with two conditions that serve to validate the context scenarios as well as the naturalness and recognizability of the video sequences. Our results provide clear evidence that conversational expressions can be recognized surprisingly well from visual information alone. The MPI facial expression database will enable researchers from different research fields (including the perceptual and cognitive sciences, but also affective computing, as well as computer vision) to investigate the processing of a wider range of natural facial expressions. no notspecified http://www.kyb.tuebingen.mpg.de/ published 17 The MPI Facial Expression Database: A Validated Database of Emotional and Conversational Facial Expressions 15017 15422 IsenbergC2011 3 T Isenberg D Cunningham 2011-12-00 8 30 2457–2458 Computer Graphics Forum no notspecified http://www.kyb.tuebingen.mpg.de/ published -2457 Computational Aesthetics 2011 in Vancouver, Canada, August 5–7, 2011, Sponsored by Eurographics, in Collaboration with ACM SIGGRAPH 15017 15422 5746 3 T Stich C Linz C Wallraven DW Cunningham M Magnor 2011-01-00 2 8 1 28 ACM Transactions on Applied Perception We present a method for image interpolation that is able to create high-quality, perceptually convincing transitions between recorded images. By implementing concepts derived from human vision, the problem of a physically correct image interpolation is relaxed to that of image interpolation which is perceived as visually correct by human observers. We find that it suffices to focus on exact edge correspondences, homogeneous regions and coherent motion to compute convincing results. A user study confirms the visual quality of the proposed image interpolation approach. We show how each aspect of our approach increases perceived quality of the result. We compare the results to other methods and assess achievable quality for different types of scenes. no notspecified http://www.kyb.tuebingen.mpg.de/ published 27 Perception-Motivated interpolation of image sequences 15017 15422 5915 3 DW Cunningham C Wallraven 2009-12-00 13:7 9 1 17 Journal of Vision Communication is critical for normal, everyday life. During a conversation, information is conveyed in a number of ways, including through body, head, and facial changes. While much research has examined these latter forms of communication, the majority of it has focused on static representations of a few, supposedly universal expressions. Normal conversations, however, contain a very wide variety of expressions and are rarely, if ever, static. Here, we report several experiments that show that expressions that use head, eye, and internal facial motion are recognized more easily and accurately than static versions of those expressions. Moreover, we demonstrate conclusively that this dynamic advantage is due to information that is only available over time, and that the temporal integration window for this information is at least 100 ms long. no notspecified http://www.kyb.tuebingen.mpg.de/ published 16 Dynamic information for the recognition of conversational expressions 15017 15422 5741 3 C Wallraven R Fleming DW Cunningham J Rigau M Feixas M Sbert 2009-08-00 4 33 484 495 Computers and Graphics The categorization of art (paintings, literature) into distinct styles such as Expressionism, or Surrealism has had a profound influence on how art is presented, marketed, analyzed, and historicized. Here, we present results from human and computational experiments with the goal of determining to which degree such categories can be explained by simple, low-level appearance information in the image. Following experimental methods from perceptual psychology on category formation, naive, non-expert participants were first asked to sort printouts of artworks from different art periods into categories. Converting these data into similarity data and running a multi-dimensional scaling (MDS) analysis, we found distinct categories which corresponded sometimes surprisingly well to canonical art periods. The result was cross-validated on two complementary sets of artworks for two different groups of participants showing the stability of art interpretation. The second focus of this paper was on determining how far computational algorithms would be able to capture human performance or would be able in general to separate different art categories. Using several state-of-the-art algorithms from computer vision, we found that whereas low-level appearance information can give some clues about category membership, human grouping strategies included also much higher-level concepts. no notspecified http://www.kyb.tuebingen.mpg.de/ published 11 Categorizing art: Comparing humans and computers 15017 15422 4599 3 M Nusseck DW Cunningham C Wallraven HH Bülthoff 2008-06-00 8:1 8 1 23 Journal of Vision The human face is an important and complex communication channel. Humans can, however, easily read in a face not only identity information, but also facial expressions with high accuracy. Here, we present the results of four psychophysical experiments in which we systematically manipulated certain facial areas in video sequences of nine conversational expressions to investigate recognition performance and its dependency on the motions of different facial parts. These studies allowed us to determine what information is {it necessary} and {it sufficient} to recognize the different facial expressions. Subsequent analyses of the face movements and correlation with recognition performance show that, for some expressions, one individual facial region can represent the whole expression. In other cases, the interaction of more than one facial area is needed to clarify the expression. The full set of results is used to develop a systematic description of the roles of different facial parts in the visual perception of conversational facial expressions. no notspecified http://www.kyb.tuebingen.mpg.de/ published 22 The contribution of different facial regions to the recognition of conversational expressions 15017 15422 5750 3 DW Cunningham 2008-04-00 2 31 203 204 Behavioural and Brain Science no notspecified http://www.kyb.tuebingen.mpg.de/ published 1 Visual Prediction as indicated by perceptual adaptation to temporal delays and discrete stimulation 3996 3 C Wallraven M Breidt DW Cunningham HH Bülthoff 2008-01-00 4:4 4 1 20 ACM Transactions on Applied Perception The human face is capable of producing an astonishing variety of expressions—expressions for which sometimes the smallest difference changes the perceived meaning considerably. Producing realistic-looking facial animations that are able to transport this degree of complexity continues to be a challenging research topic in computer graphics. One important question that remains to be answered is: When are facial animations good enough? Here we present an integrated framework in which psychophysical experiments are used in a first step to systematically evaluate the perceptual quality of several different computer-generated animations with respect to real-world video sequences. The first experiment provides an evaluation of several animation techniques, exposing specific animation parameters that are important to achieve perceptual fidelity. In a second experiment we then use these benchmarked animation techniques in the context of perceptual research in order to systematically investigate the spatio-temporal characteristics of expressions. A third and final experiment uses the quality measures that were developed in the first two experiments to examine the perceptual impact of changing facial features to improve the animation techniques. Using such an integrated approach, we are able to provide important insights into facial expressions for both the perceptual and computer graphics community. no notspecified http://www.kyb.tuebingen.mpg.de/ published 19 Evaluating the Perceptual Realism of Animated Facial Expressions 15017 15422 4592 3 C Wallraven HH Bülthoff J Fischer DW Cunningham D Bartz 2007-11-00 3:16 4 1 24 ACM Transactions on Applied Perception The goal of stylization is to provide an abstracted representation of an image that highlights specific types of visual information. Recent advances in computer graphics techniques have made it possible to render many varieties of stylized imagery efficiently making stylization into a useful technique, not only for artistic, but also for visualization applications. In this paper, we report results from two sets of experiments that aim at characterizing the perceptual impact and effectiveness of three different stylization techniques in the context of dynamic facial expressions. In the first set of experiments, animated facial expressions are stylized using three common techniques (brush, cartoon, and illustrative stylization) and investigated using different experimental measures. Going beyond the usual questionnaire approach, these experiments compare the techniques according to several criteria ranging from subjective preference to task-dependent measures (such as recognizability, intensity) allowing us to compare behavioral and introspective approaches. The second set of experiments use the same stylization techniques on real-world video sequences in order to compare the effect of stylization on natural and artificial stimuli. Our results shed light on how stylization of image contents affects the perception and subjective evaluation of both real and computer-generated facial expressions. no notspecified http://www.kyb.tuebingen.mpg.de/ published 23 Evaluation of Real-World and Computer-Generated Stylized Facial Expressions 15017 15422 3768 3 BE Riecke DW Cunningham HH Bülthoff 2006-09-00 3 71 298 313 Psychological Research Robust and effortless spatial orientation critically relies on “automatic and obligatory spatial updating”, a largely automatized and reflex-like process that transforms our mental egocentric representation of the immediate surroundings during ego-motions. A rapid pointing paradigm was used to assess automatic/obligatory spatial updating after visually displayed upright rotations with or without concomitant physical rotations using a motion platform. Visual stimuli displaying a natural, subject-known scene proved sufficient for enabling automatic and obligatory spatial updating, irrespective of concurrent physical motions. This challenges the prevailing notion that visual cues alone are insufficient for enabling such spatial updating of rotations, and that vestibular/proprioceptive cues are both required and sufficient. Displaying optic flow devoid of landmarks during the motion and pointing phase was insufficient for enabling automatic spatial updating, but could not be entirely ignored either. Interestingly, additional physical motion cues hardly improved performance, and were insufficient for affording automatic spatial updating. The results are discussed in the context of the mental transformation hypothesis and the sensorimotor interference hypothesis, which associates difficulties in imagined perspective switches to interference between the sensorimotor and cognitive (to-be-imagined) perspective. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/Riecke__06_PsychologicalResearch_onlinePublication__Spatial_Updating_in_Virtual_Reality_-_The_Sufficiency_of_Visual_Information_3768[0].pdf published 15 Spatial updating in virtual reality: the sufficiency of visual information 15017 15422 3540 3 D Cunningham M Kleiner C Wallraven H Bülthoff 2005-07-00 3 2 251 269 ACM Transactions on Applied Perception Communication plays a central role in everday life. During an average conversation, information is exchanged in a variety of ways, including through facial motion. Here, we employ a custom, model-based image manipulation technique to selectively "freeze" portions of a face in video recordings in order to determine the areas that are sufficient for proper recognition of nine conversational expressions. The results show that most expressions rely primarily on a single facial area to convey meaning, with different expressions using different areas. The results also show that already the combination of rigid head, eye, eyebrow, and mouth motions is sufficient to produce expressions that are as easy to recognize as the original, unmanipulated recordings. Finally, the results show that the manipulation technique introduced few perceptible artifacts into the altered video sequences. This fusion of psychophysics and computer graphics techniques provides not only fundamental insights into human perception and cognitio n, but also yields the basis for a systematic description of what needs to move in order to produce realistic, recognizable conversational facial animations. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/cunningham-etal-ACM-tap-2005_3540[0].pdf published 18 Manipulating video sequences to determine the components of conversational facial expressions 15017 15422 2867 3 D Cunningham M Nusseck C Wallraven HH Bülthoff 2004-07-00 3-4 15 305 310 Computer Animation & Virtual Worlds Facial expressions can be used to direct the flow of a conversation as well as to improve the clarity of communication. The critical physical differences between expressions can, however, be small and subtle. Clear presentation of facial expressions in applied settings, then, would seem to require a large conversational agent. Given that visual displays are generally limited in size, the usage of a large conversational agent would reduce the amount of space available for the display of other information. Here, we examine the role of image size in the recognition of facial expressions. The results show that conversational facial expressions can be easily recognized at surprisingly small image sizes. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf2867.pdf published 5 The role of image size in the recognition of conversational facial expressions 15017 15422 2335 3 JP de Ruiter S Rossignol L Vuurpijl DW Cunningham WJM Levelt 2003-00-00 3 35 408 419 Behavior Research Methods, Instruments, & Computers no notspecified http://www.kyb.tuebingen.mpg.de/ published 11 SLOT: A research platform for investigating multimodal communication 1827 3 WO Readinger A Chatziastros DW Cunningham HH Bülthoff JE Cutting 2002-12-00 4 8 247 258 Journal of Experimental Psychology: Applied The effects of gaze-eccentricity on the steering of an automobile were studied. Drivers performed an attention task while attempting to drive down the middle of a straight road in a simulation. Steering was biased in the direction of fixation and deviation from the center of the road was proportional to the gaze direction until saturation at approximately 15 degrees gaze-angle from straight ahead. This effect remains when the position of the head was controlled and a reverse-steering task was used. Furthermore, the effect was not dependent upon speed, but reversed when the forward movement of the driver was removed from the simulation. Thus, small deviations in a driver's gaze can lead to significant impairments of the ability to drive a straight course. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf1827.pdf published 11 Gaze-eccentricity effects on road position and steering 15017 15422 34 3 DW Cunningham A Chatziastros M von der Heyde HH Bülthoff 2001-11-00 2:3 1 88 98 Journal of Vision Rapid and accurate visuomotor coordination requires tight spatial and temporal sensorimotor synchronization. The introduction of a sensorimotor or intersensory misalignment (either spatial or temporal) impairs performance on most tasks. For more than a century, it has been known that a few minutes of exposure to a spatial misalignment can induce a recalibration of sensorimotor spatial relationships, a phenomenon that may be referred to as spatial visuomotor adaptation. Here, we use a high-fidelity driving simulator to demonstrate that the sensorimotor system can adapt to temporal misalignments on very complex tasks, a phenomenon that we refer to as temporal visuomotor adaptation. We demonstrate that adapting on a single street produces an adaptive state that generalizes to other streets. This shows that temporal visuomotor adaptation is not specific to a single visuomotor transformation, but generalizes across a class of transformations. Temporal visuomotor adaptation is strikingly parallel to spatial visuomotor adaptation, and has strong implications for the understanding of visuomotor coordination and intersensory integration. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf34.pdf published 10 Driving in the future: Temporal visuomotor adaptation and generalization 15017 15422 1240 3 DW Cunningham VA Billock BH Tsou 2001-11-00 6 12 532 535 Psychological Science Most events are processed by a number of neural pathways. These pathways often differ considerably in processing speed. Thus, coherent perception requires some form of synchronization mechanism. Moreover, this mechanism must be flexible, because neural processing speed changes over the life of an organism. Here we provide behavioral evidence that humans can adapt to a new intersensory temporal relationship (which was artificially produced by delaying visual feedback). The conflict between these results and previous work that failed to find such improvements can be explained by considering the present results as a form of sensorimotor adaptation. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf1240.pdf published 3 Sensorimotor Adaptation to Violations of Temporal Contiguity 15017 15422 1238 3 VA Billock DW Cunningham PR Havig BH Tsou 2001-10-00 10 18 2404 2413 Journal of the Optical Society of America A Recent work establishes that static and dynamic natural images have fractal-like 1/f spatiotemporal spectra. Artifical textures, with randomized phase spectra, and 1/f amplitude spectra are also used in studies of texture and noise perception. Influenced by colorimetric principles and motivated by the ubiquity of 1/f spatial and temporal image spectra, we treat the spatial and temporal frequency exponents as the dimensions characterizing a dynamic texture space, and we characterize two key attributes of this space, the spatiotemporal appearance map and the spatiotemporal discrimination function (a map of MacAdam-like just-noticeable-difference contours). no notspecified http://www.kyb.tuebingen.mpg.de/ published 9 Perception of spatiotemporal random fractals: an extension of colorimetric methods to the study of dynamic texture 1239 3 DW Cunningham 2001-08-00 3-4 20 209 213 Cahiers de Psychologie Cognitive no notspecified http://www.kyb.tuebingen.mpg.de/ published 4 The central role of time in an identity decision theory 15017 15422 1242 3 D Field TF Shipley DW Cunningham 1999-00-00 61(1) 161 176 Perception and Psychophysics no notspecified http://www.kyb.tuebingen.mpg.de/ published 15 Prism Adaptation to dynamic events. 1244 3 DW Cunningham TF Shipley PJ Kellman 1998-00-00 60(5) 839 851 Perception and Psychophysics no notspecified http://www.kyb.tuebingen.mpg.de/ published 12 Interactions between spatial and spatiotemporal information in Spatiotemporal Boundary Formation. 1243 3 DW Cunningham TF Shipley PJ Kellman 1998-00-00 27(4) 403 416 Perception no notspecified http://www.kyb.tuebingen.mpg.de/ published 13 The dynamic specification of surfaces and boundaries. CunninghamW2013 7 DW Cunningham C Wallraven Girona, Spain2013-05-00 34th Annual Conference of the European Association for Computer Graphics Humans and computer both have limited resources with which they must process the massive amount of information present in the natural world. For over 150 years, physiologists and psychologists have been performing experiments to elucidate what information humans and animals can detect as well as how they extract, represent and process that information. Recently, there has been an increasing trend of computer scientists performing similar experiments, although often with quite different goals. This tutorial will provide a basic background on the design and execution of perceptual experiments for the practicing computer scientist. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 Understanding and Designing Perceptual Experiments 15017 15422 656 7 W Readinger A Chatziastros D Cunningham HH Bülthoff JE Cutting Brisbane, Australia2012-00-00 321 326 9th International Conference Vision in Vehicles (VIV 2001) Instructions given to novices learning certain tasks of applied navigation often suggest that gazedirection (?line of sight?) should preview the path the operator desires to take (e.g., Bondurant & Blakemore, 1998; Motorcycle Safety Foundation, 1992; Morris, 1990), presumably because looking behavior can ultimately affect steering control through hand, arm, or leg movements that could lead to undesired path deviations. Here, we control participants? gaze-direction while driving an automobile in virtual reality, and find that gaze-eccentricity has a large, systematic effect on steering and lane-position. Moreover, even when head-position and postural effects of the driver are controlled, there remains a significant bias to drive in the direction of fixation, indicating the existence of a perceptual, and not merely motor, phenomenon. no notspecified http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/VIV-2001-Readinger.pdf published 5 Gaze-direction and steering effects while driving 15017 15422 5944 7 Z Salah DW Cunningham D Bartz Berlin, Germany2009-09-00 311 316 Workshop der Tagung Mensch & Computer 2009 Illustrationen werden erfolgreich in den Ingenieurwissenschaften, den Naturwissenschaften und der Medizin zur abstrahierten Darstellung von Objekten und Situationen verwendet. Typischerweise sind diese Illustrationen Zeichnungen, bei denen der Illustrator künstlerische Techniken zur Betonung relevanter Aspekte der Objekte einsetzt. Im Gegensatz dazu erzeugen Visualisierungen eine direkte, nicht abstrahierte visuelle Darstellung von Simulationen, gescannten Objekten oder modellierten Daten. Durch die inhärente Komplexität dieser Datensätze stellt sich die Interpretation dieser Daten jedoch oft als schwierig dar. Die illustrative Visualisierung hingegen versucht beide Ansätze zur einer abstrahierten Darstellung eines Datensatzes zu verbinden, in der die wesentlichen Charakteristika betont werden. Dieser Ansatz bekommt eine besondere Bedeutung bei sehr komplexen Modellen, die u.U. aus vielen einzelnen Objekten bestehen, wie z.B. einzelne Bauteile einer Maschine, oder segmentierten Organen aus einem CT- oder MRT-Datensatz eines Menschen. Während im Allgemeinen die illustrative Visualisierung einer bessere Betonung ausgewählter und relevanter Informationen als die traditionelle Visualisierung erreicht, so stellen viele nah beieinander gelegene Objekte eine Herausforderung dar, da sie klar von einander getrennt werden müssen. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/MenschUndComputer2009-Salah_5944[0].pdf published 5 Perzeptuell motivierte illustrative Darstellungsstile für komplexe Modelle 15017 15422 5936 7 DW Cunningham C Wallraven Chania, Crete, Greece2009-09-00 41 44 6th Symposium on Applied Perception in Graphics and Visualization (APGV 2009) Faces are a powerful and versatile communication channel. Physically, facial expressions contain a considerable amount of information, yet it is clear from stylized representations such as cartoons that not all of this information needs to be present for efficient processing of communicative intent. Here, we use a high-fidelity facial animation system to investigate the importance of two forms of spatial information (connectivity and the number of vertices) for the perception of intensity and the recognition of facial expressions. The simplest form of connectivity is point light faces. Since they show only the vertices, the motion and configuration of features can be seen but the higher-frequency spatial deformations cannot. In wireframe faces, additional information about spatial configuration and deformation is available. Finally, full-surface faces have the highest degree of static information. The results of two experiments are presented. In the first, the presence of motion was manipulated. In the second, the size of the images was varied. Overall, dynamic expressions performed better than static expressions and were largely impervious to the elimination of shape or connectivity information. Decreasing the size of the image had little effect until a critical size was reached. These results add to a growing body of evidence that shows the critical importance of dynamic information for processing of facial expressions: As long as motion information is present, very little spatial information is required. no notspecified http://www.kyb.tuebingen.mpg.de/ published 3 The interaction between motion and form in expression recognition 15017 15422 5740 7 C Wallraven DW Cunningham J Rigau M Feixas M Sbert Victoria, BC, Canada2009-05-00 137 144 Eurographics Workshop on Computational Aesthetics in Graphics, Visualization and Imaging By looking at a work of art, an observer enters into a dialogue. In this work, we attempt to analyze this dialogue with both behavioral and computational tools. In two experiments, observers were asked to look at a large number of paintings from different art periods and to rate their visual complexity, or their aesthetic appeal. During these two tasks, their eye movements were recorded. The complexity and aesthetic ratings show clear preferences for certain artistic styles and were based on both low-level and high-level criteria. Eye movements reveal the time course of the aesthetic dialogue as observers try to interpret and understand the painting. Computational analyses of both the ratings (using measures derived from information theory) and the eye tracking data (using two models of saliency) showed that our computational tools are already able to explain some properties of this dialogue. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/CAe2009-Wallraven_5740[0].pdf published 7 Aesthetic appraisal of art: from eye movements to computers 15017 15422 5164 7 T Stich C Linz C Wallraven DW Cunningham M Magnor Los Angeles, CA, USA2008-08-00 97 106 5th Symposium on Applied Perception in Graphics and Visualization (APGV 2008) We present a method for image interpolation which is able to create high-quality, perceptually convincing transitions between recorded images. By implementing concepts derived from human vision, the problem of a physically correct image interpolation is relaxed to an image interpolation that is perceived as physically correct by human observers. We find that it suffices to focus on exact edge correspondences, homogeneous regions and coherent motion to compute such solutions. In our user study we confirm the visual quality of the proposed image interpolation approach. We show how each aspect of our approach increases the perceived quality of the interpolation results, compare the results obtained by other methods and investigate the achieved quality for different types of scenes. no notspecified http://www.kyb.tuebingen.mpg.de/ published 9 Perception-motivated interpolation of image sequences 15017 15422 5163 7 C Wallraven DW Cunningham R Fleming Lisboa, Portugal2008-06-00 131 138 Computational Aesthetics 2008: Eurographics Workshop on Computational Aesthetics (CAe 2008) The categorization of art (paintings, literature) into distinct styles such as expressionism, or surrealism has had a profound influence on how art is presented, marketed, analyzed, and historicized. Here, we present results from several perceptual experiments with the goal of determining whether such categories also have a perceptual foundation. Following experimental methods from perceptual psychology on category formation, naive, non-expert participants were asked to sort printouts of artworks from different art periods into categories. Converting these data into similarity data and running a multi-dimensional scaling (MDS) analysis, we found distinct perceptual categories which did in some cases correspond to canonical art periods. Initial results from a comparison with several computational algorithms for image analysis and scene categorization are also reported. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/CAE2008-Wallraven_5163[0].pdf published 7 Perceptual and Computational Categories in Art 15017 15422 5161 7 D Bartz DW Cunningham J Fischer C Wallraven Creta, Greece2008-04-00 65 86 29th Annual Conference of the European Association for Computer Graphics (EG 2008) Traditionally, computer graphics strived to achieve the technically best representation of the scenario or scene. For rendering, this lead to the preeminence of representations based on the physics of light interacting with different media and materials. Research in virtual reality has focused on interactivity and therefore on real-time rendering techniques that improve the immersion of users in the virtual environments. In contrast, visualization has focused on representations that that maximizes the information content. In most cases, such representations are not physically-based, requiring instead more abstract approaches. Recently, the increasing integration of the extensive knowledge and methods from perception research into computer graphics has fundamentally altered both fields, offering not only new research questions, but also new ways of solving existing issues. In rendering, for example, the integration can lead to the targeted allocation of computing resources to aspects of a scene that matter mos t for human observers. In visualization, the manner in which information is presented is now often driven by knowledge of low-level cues (e.g., pre-attentive features). Assumptions about how to best present information are evaluated by a psychophysical experiment. This same trend towards perceptually driven research has perhaps had the longest tradition in virtual reality, where the user&amp;amp;#8217;s response to specific interaction and rendering techniques is examined using a variety of methods. Against this backdrop of an increasing importance of perceptual research in all areas related to computer generated imagery, we provide a state of the art report on the current state of perception in computer graphics. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/Eurographics08-Bartz_5161[0].pdf published 21 The Role of Perception for Computer Graphics 15017 15422 5497 7 M Nusseck DW Cunningham JPD Ruiter HH Bülthoff Hilvarenbeek, Netherlands2007-09-00 1 6 International Conference on Auditory-Visual Speech Processing 2007 (AVSP2007) Multimodal prosody carries a wide variety of information Here, we investigated the roles of visual and the auditory information in the production and perception of different emphasis intensities. In a series of video recordings, the intensity, location, and syntactic category of the emphasized word were varied. Physical analyses demonstrated that each speaker produced different emphasis intensities, with a high degree of individual variation in information distribution. In the first psychophysical experiment, observers easily distinguished between the different intensities. Interestingly, the pattern of perceived intensity was remarkably similar across speakers, despite the individual variations in the use of different visual and acoustic modalities. The second experiment presented the recordings visually, acoustically, and audiovisually. Overall, while the audio only condition was very similar to the audiovisual condition, there was a clear influence of visual information. Weak visual information lead to a weaker audiovisual intensity, while stong visual information enhanced audiovisual intensity. no notspecified http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/AVSP-2007-Nusseck.pdf published 5 Perception of Prominence Intensity in audio-visual Speech 15017 15422 4465 7 RT Griesser DW Cunningham C Wallraven HH Bülthoff Tübingen, Germany2007-07-00 11 18 4th Symposium on Applied Perception in Graphics and Visualization (APGV 2007) The human face is capable of producing a large variety of facial expressions that supply important information for communication. As was shown in previous studies using unmanipulated video sequences, movements of single regions like mouth, eyes, and eyebrows as well as rigid head motion play a decisive role in the recognition of conversational facial expressions. Here, flexible but at the same time realistic computer animated faces were used to investigate the spatiotemporal coaction of facial movements systematically. For three psychophysical experiments, spatiotemporal properties were manipulated in a highly controlled manner. First, single regions (mouth, eyes, and eyebrows) of a computer animated face performing seven basic facial expressions were selected. These single regions, as well as combinations of these regions, were animated for each of the seven chosen facial expressions. Participants were then asked to recognize these animated expressions in the experiments. The findings show that the animated avatar in general is a useful tool for the investigation of facial expressions, although improvements have to be made to reach a higher recognition accuracy of certain expressions. Furthermore, the results shed light on the importance and interplay of individual facial regions for recognition. With this knowledge the perceptual quality of computer animations can be improved in order to reach a higher level of realism and effectiveness. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/apgv07-11_4465[0].pdf published 7 Psychophysical investigation of facial expressions using computer animated faces 15017 15422 4467 7 DW Cunningham C Wallraven RW Fleming W Strasser Banff, Alberta, Canada2007-06-00 89 96 Eurographics Workshop on Computational Aesthetics in Graphics, Visualization and Imaging (CAe '07) The recent increase in both the range and the subtlety of computer graphics techniques has greatly expanded the possibilities for synthesizing images. In many cases, however, the relationship between the parameters of an algorithm and the resulting perceptual effect is not straightforward. Since the ability to produce specific, intended effects is a natural pre-requisite for many scientific and artistic endeavors, this is a strong drawback. Here, we demonstrate a generalized method for determining both the qualitative and quantitative mapping between parameters and perception. Multidimensional Scaling extracts the metric structure of perceived similarity between the objects, as well as the transformation between similarity space and parameter space. Factor analysis of semantic differentials is used to determine the aesthetic structure of the stimulus set. Jointly, the results provide a description of how specific parameter changes can produce specific semantic changes. The method is demonstrated using two datasets. The first dataset consisted of glossy objects, which turned out to have a 2D similarity space and five primary semantic factors. The second dataset, transparent objects, can be described with a non-linear, 1D similarity map and six semantic factors. In both cases, roughly half of the factors represented aesthetic aspects of the stimuli, and half the low-level material properties. Perceptual reparameterization of computer graphics algorithms (such as those dealing with the representation of surface properties) offers the potential to improve their accessibility. This will not only allow easier generation of specific effects, but also enable more intuitive exploration of different image properties. no notspecified http://www.kyb.tuebingen.mpg.de/ published 7 Perceptual reparameterization of material properties 15017 15422 3984 7 C Wallraven J Fischer DW Cunningham D Bartz HH Bülthoff Boston, MA, USA2006-07-00 85 92 3rd Symposium on Applied Perception in Graphics and Visualization (APGV 2006) Stylized rendering aims to abstract information in an image making it useful not only for artistic but also for visualization purposes. Recent advances in computer graphics techniques have made it possible to render many varieties of stylized imagery efficiently. So far, however, few attempts have been made to characterize the perceptual impact and effectiveness of stylization. In this paper, we report several experiments that evaluate three different stylization techniques in the context of dynamic facial expressions. Going beyond the usual questionnaire approach, the experiments compare the techniques according to several criteria ranging from introspective measures (subjective preference) to task-dependent measures (recognizability, intensity). Our results shed light on how stylization of image contents affects the perception and subjective evaluation of facial expressions. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/p85-wallraven_3984[0].pdf published 7 The Evaluation of Stylized Facial Expressions 15017 15422 3876 7 J Fischer D Cunningham D Bartz C Wallraven HH Bülthoff W Strasser Lisboa, Portugal2006-05-00 53 61 12. Eurographics Symposium on Virtual Environments (EGVE 06) In augmented reality, virtual graphical objects are overlaid over the real environment of the observer. Conventional augmented reality systems normally use standard real-time rendering methods for generating the graphical representations of virtual objects. These renderings contain the typical artifacts of computer generated graphics, e.g., aliasing caused by the rasterization process and unrealistic, manually configured illumination models. Due to these artifacts, virtual objects look artifical and can easily be distinguished from the real environment. A different approach to generating augmented reality images is the basis of stylized augmented reality [FBS05c]. Here, similar types of artistic or illustrative stylization are applied to the virtual objects and the camera image of the real enviroment. Therefore, real and virtual image elements look significantly more similar and are less distinguishable from each other. In this paper, we present the results of a psychophysical study on the effectiveness of stylized augmented reality. In this study, a number of participants were asked to decide whether objects shown in images of augmented reality scenes are virtual or real. Conventionally rendered as well as stylized augmented reality images and short video clips were presented to the participants. The correctness of the participants' responses and their reaction times were recorded. The results of our study show that an equalized level of realism is achieved by using stylized augmented reality, i.e., that it is significantly more difficult to distinguish virtual objects from real objects. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/Fischer-2006-Measuring_3876[0].pdf published 8 Measuring the Discernability of Virtual Objects in Conventional and Stylized Augmented Reality 15017 15422 3913 7 DW Cunningham H-G Nusseck H Teufel C Wallraven HH Bülthoff Alexandria, VA, USA2006-03-00 111 118 IEEE Virtual Reality Conference 2006 Virtual Reality (VR) is increasingly being used in industry, medicine, entertainment, education, and research. It is generally critical that the VR setups produce behavior that closely resembles real world behavior. One part of any task is the ability to control our posture. Since postural control is well studied in the real world and is known to be strongly influenced by visual information, it is an ideal metric for examining the behavioral fidelity of VR setups. Moreover, VR-based experiments on postural control can provide fundamental new insights into human perception and cognition. Here, we employ the "swinging room paradigm" to validate a specific VR setup. Furthermore, we systematically examined a larger range of room oscillations than previously studied in any single setup. We also introduce several new methods and analyses that were specifically designed to optimize the detection of synchronous swinging between the observer and the virtual room. The results show that the VR setup has a very high behavioral fidelity and that increases in swinging room amplitude continue to produce increases in body sway even at very large room displacements (+/- 80 cm). Finally, the combination of new methods proved to be a very robust, reliable, and sensitive way of measuring body sway. no notspecified http://www.kyb.tuebingen.mpg.de/ published 7 A psychophysical examination of Swinging Rooms, Cylindrical Virtual Reality setups, and characteristic trajectories 15017 15422 3541 7 C Wallraven M Breidt D Cunningham HH Bülthoff La Coruña, Spain2005-08-00 17 24 2nd Symposium on Applied Perception in Graphics and Visualization (APGV 2005) The human face is capable of producing an astonishing variety of expressions - expressions for which sometimes the smallest difference changes the perceived meaning noticably. Producing realistic-looking facial animations that are able to transport this degree of complexity continues to be a challenging research topic in computer graphics. One important question that remains to be answered is: When are facial animations good enough? Here we present an integrated framework in which psychophysical experiments are used in a first step to systematically evaluate the perceptual quality of computer-generated animations with respect to real-world video sequences. The result of the first experiment is an evaluation of several animation techniques in which we expose specific animation parameters that are important for perceptual fidelity. In a second experiment we then use these benchmarked animations in the context of perceptual research in order to systematically investigate the spatio-temporal characteristics of ex pressions. Using such an integrated approach, we are able to provide insights into facial expressions for both the perceptual and computer graphics community. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/wallraven-etal-apgv-2005_[0].pdf published 7 Psychophysical evaluation of animated facial expressions 15017 15422 5979 7 B Adelstein HH Bülthoff DW Cunningham K Mania N Mourkoussis E Swan N Thalmann T Troscianko Bonn, Germany2005-03-00 308 IEEE Conference on Virtual Reality (VR '05) It is increasingly important to provide fidelity mecrics for rendered images and interactive virtual environments (VEs) targeting transfer of training in real-world task situations. Computational metrics which aim to predict the degree of fidelity of a rendered image can be based on psychophysical observations. For interactive simulations, psychophysical investigations can be carried out into the degree of similarity between the original and a synthetic simulation. Psychophysics comprises a collection of methods used to conduct non-invasive experiments on humans, the purpose of which is to study mappings between events in an environment and levels of sensory responses to those events. This tutorial will present the techniques and principles towards conducting psychophysical studies, for assessing image quality as well as fidelity of a VE simulation and how results from such studies contribute to VE system design as well as to computational image quality metrics. Methods based on experiments for measuring the perceptual equivalence between a real scene and a computer simulation of the same scene will be reported. These methods are presented through the study of human vision and include using photorealistic computer graphics to depict complex environments and works of art. In addition, physical and psychophysical fidelity issues in the assessment of virtual environments will be emphasised. Specifications for correct matching between the psychophysical characteristics of the displays and the human users’ sensory and motor systems will he discussed as well as some examples of the consequences when systems fail to be physically well matched to their users. Complete experimental cycles will be described from the initial idea and design, to pilot study, experimental redesign, data collection, analysis and post-experiment lessons learned. Examples will include research on spatial orientation in Virtual Reality, assessing fidelity of flight simulators, fidelity of simulation of humans and clothing and measuring perceptual sensitivity to latency. This tutorial requires no fundamental prerequisites. It would help if the attendee had some knowledge of experimental design, and of some personal experience of computer graphics and simulation systems. However, the course will be self-contained. no notspecified http://www.kyb.tuebingen.mpg.de/ published -308 Human-Centred Fidelity Metrics for Virtual Environment Simulations 15017 15422 3058 7 M Kleiner C Wallraven M Breidt DW Cunningham HH Bülthoff Zermatt, Switzerland2004-12-00 55 60 Workshop on Modelling and Motion Capture Techniques for Virtual Environments (CAPTECH 2004) In order to produce realistic-looking avatars, computer graphics has traditionally relied solely on physical realism. Research on cognitive aspects of face perception, however, can provide insights into how to produce believable and recognizable faces. In this paper, we describe a method for automatically manipulating video recordings of faces. The technique involves the use of a custom-built multi-viewpoint video capture system in combination with head motion tracking and a detailed 3D head shape model. We illustrate how the technique can be employed in studies on dynamic facial expression perception by summarizing the results of two psychophysical studies which provide suggestions for creating recognizable facial expressions. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf3058.pdf published 5 Multi-viewpoint video capture for facial perception research 15017 15422 2791 7 T Cooke DW Cunningham HH Bülthoff Tübingen, Germany2004-09-00 407 414 26th Annual Symposium of the German Association for Pattern Recognition (DAGM 2004) When an object moves, it covers and uncovers texture in the background. This pattern of change is sufficient to define the object’s shape, velocity, relative depth, and degree of transparency, a process called Spatiotemporal Boundary Formation (SBF). We recently proposed a mathematical framework for SBF, where texture transformations are used to recover local edge segments, estimate the figure’s velocity and then reconstruct its shape. The model predicts that SBF should be sensitive to spatiotemporal noise, since the spurious transformations will lead to the recovery of incorrect edge orientations. Here we tested this prediction by adding a patch of dynamic noise (either directly over the figure or a fixed distance away from it). Shape recognition performance in humans decreased to chance levels when noise was placed over the figure but was not affected by noise far away. These results confirm the model’s prediction and also imply that SBF is a local process. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf2791.pdf published 7 The Perceptual Influence of Spatiotemporal Noise on the Reconstruction of Shape from Dynamic Occlusion 15017 15422 2865 7 DW Cunningham M Kleiner C Wallraven HH Bülthoff Los Angeles, CA, USA2004-08-00 143 149 1st Symposium on Applied Perception in Graphics and Visualization (APGV 2004) Conversing with others is one of the most central of human behaviours. In any conversation, humans use facial motion to help modify what is said, to control the flow of a dialog, or to convey complex intentions without saying a word. Here, we employ a custom, image-based, stereo motion-tracking algorithm to track and selectively "freeze" portions of an actor or actress's face in video recordings in order to determine the necessary and sufficient facial motions for nine conversational expressions. The results show that most expressions rely primarily on a single facial area to convey meaning, with different expressions using different facial areas. The results also show that the combination of rigid head, eye, eyebrow, and mouth motion is sufficient to produce versions of these expressions that are as easy to recognize as the original recordings. Finally, the results show that the manipulation technique introduced few perceptible artifacts into the altered video sequences. The use of advanced computer graphics techniques provided a means to systematically examine real facial expressions. This provides not only fundamental insights into human perception and cognition, but also yields the basis for a systematic description of what needs to be animated in order to produce realistic, recognizable facial expressions. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf2865.pdf published 6 The components of conversational facial expressions 15017 15422 2808 7 M Kleiner A Schwaninger DW Cunningham B Knappmeyer Los Angeles, CA, USA2004-08-00 180 1st Symposium on Applied Perception in Graphics and Visualization (APGV 2004) Manipulated still images of faces have often been used as stimuli for psychophysical research on human perception of faces and facial expressions. In everyday life, however, humans are usually confronted with moving faces. We describe an automated way of performing manipulations on facial video recordings and how it can be applied to investigate human dynamic face perception. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf2808.pdf published -180 Using facial texture manipulation to study facial motion perception 15017 15422 2866 7 C Wallraven DW Cunningham M Breidt HH Bülthoff Los Angeles, CA, USA2004-08-00 181 1st Symposium on Applied Perception in Graphics and Visualization (APGV 2004) The viewpoint dependency of complex facial expressions versus simple facial motions was analyzed. The MPI Tübingen Facial Expression Database was used for the psychophysical investigation of view dependency. The ANOVA revealed at best marginally significant effects of viewpoint or type of expression on recognition accuracy. It was observed that humans were able to recognize facial motions in a largely viewpoint invariant manner, which supports the theoretical model of face recognition. It was also suggested that in order to be recognized, computer generated facial expressions should 'look good' from all viewpoints. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/apgv04-181_2866[0].pdf published -181 View dependence of complex versus simple facial motions 15017 15422 2320 7 M Breidt C Wallraven DW Cunningham HH Bülthoff Granada, Spain2003-09-00 63 66 24th Annual Conference of the European Association for Computer Graphics We present ongoing work on the development of new methods for highly realistic facial animation. One of the main contributions is the use of real-world, high-precision data for both the timing of the animation and the deformation of the face geometry. For animation, a set of morph shapes acquired through a 3D scanner is linearly morphed according to timing extracted from point tracking data recorded with an optical motion capture system. no notspecified http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/Eurographics-2003-Breidt.pdf published 3 Combining 3D Scans and Motion Capture for Realistic Facial Animation 15017 15422 2096 7 DW Cunningham M Breidt M Kleiner C Wallraven HH Bülthoff Benalmádena, Spain2003-09-00 7 12 3rd IASTED International Conference on Visualization, Imaging, and Image Processing (VIIP 2003) Since conversation is a central human activity, the synthesis of proper conversational behavior for Virtual Humans will become a critical issue. Facial expressions represent a critical part of interpersonal communication. Even with the most sophisticated, photo-realistic head model, an avatar who's behavior is unbelievable or even uninterpretable will be an inefficient or possibly counterproductive conversational partner. Synthesizing expressions can be greatly aided by a detailed description of which facial motions are perceptually necessary and sufficient. Here, we recorded eight core expressions from six trained individuals using a method-acting approach. We then psychophysically determined how recognizable and believable those expressions were. The results show that people can identify these expressions quite well, although there is some systematic confusion between particular expressions. The results also show that people found the expressions to be less than convincing. The pattern of confusions and believability ratings demonstrates that there is considerable variation in natural expressions and that even real facial expressions are not always understood or believed. Moreover, the results provide the ground work necessary to begin a more fine-grained analysis of the core components of these expressions. As some initial results from a model-based manipulation of the image sequences shows, a detailed description of facial expressions can be an invaluable aid in the synthesis of unambiguous and believable Virtual Humans. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf2096.pdf published 5 The inaccuracy and insincerity of real faces 15017 15422 2022 7 DW Cunningham M Breidt M Kleiner C Wallraven HH Bülthoff New Brunswick, NJ, USA2003-05-00 23 29 16th International Conference on Computer Animation and Social Agents (CASA 2003) Regardless of whether the humans involved are virtual or real, well-developed conversational skills are a necessity. The synthesis of interface agents that are not only understandable but also believable can be greatly aided by knowledge of which facial motions are perceptually necessary and sufficient for clear and believable conversational facial expressions. Here, we recorded several core conversational expressions (agreement, disagreement, happiness, sadness, thinking, and confusion) from several individuals, and then psychophysically determined the perceptual ambiguity and believability of the expressions. The results show that people can identify these expressions quite well, although there are some systematic patterns of confusion. People were also very confident of their identifications and found the expressions to be rather believable. The specific pattern of confusions and confidence ratings have strong implications for conversational animation. Finally, the present results provide the information necessary to begin a more fine-grained analysis of the core components of these expressions. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf2022.pdf published 6 How believable are real faces? Towards a perceptual basis for conversational animation 15017 15422 2033 7 C Kaernbach L Munka DW Cunningham Bochum, Germany2002-11-00 177 182 Workshop of GI Section 1.0.4 "Image Understanding" and the European Networks MUHCI and ECOVISION The present contribution studies the rapid adaptation process of the visuomotor system to optical transformations (here: shifting the image horizon-tally via prism goggles). It is generally believed that this adaptation consists primarily of recalibrating the transformation between visual and proprioceptive perception. According to such a purely perceptual account of adaptation, the exact path used to reach the object should not be important. If, however, it is the transformation from perception to action that is being altered, then the adapta-tion should depend on the motion trajectory. In experiments with a variety of different motion trajectories we show that visuomotor adaptation is not merely a perceptual recalibration. The structure of the motion (starting position, trajec-tory, end position) plays a central role, and even the weight load seems to be important. These results have strong implications for all models of visuomotor adaptation. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/Dynamic-Perception-2002-Cunningham.pdf published 5 Visuomotor adaptation: Dependency on motion trajectory 15017 15422 1245 7 TF Shipley DW Cunningham PJ Kellman Vancouver, Canada1993-08-00 279 283 Seventh International Conference on Event Perception and Action no notspecified http://www.kyb.tuebingen.mpg.de/ published 4 Spatiotemporal Stereopsis 15017 15422 BulthoffCW2017 2 HH Bülthoff DW Cunningham C Wallraven Springer London, UK 2011-00-00 575 596 Handbook of Face Recognition In this chapter, we will focus on the role of motion in identity and expression recognition in human, and its developmental and neurophysiological aspects. Based on results from literature, we make it clear that there is some form of characteristic facial information that is only available over time, and that it plays an important role in the recognition of identity, expression, speech, and gender; and that the addition of dynamic information improves the recognizability of expressions and identity, and can compensate for the loss of static information. Moreover, at least several different types of motion seem to exist, they play different roles, and a simple rigid/nonrigid dichotomy is neither sufficient nor appropriate to describe these motions. Additional research is necessary to determine what the dynamic features for face processing are. no notspecified http://www.kyb.tuebingen.mpg.de/ published 21 Dynamic Aspects of Face Processing in Humans 15017 15422 5749 2 VA Billock DW Cunningham BH Tsou Ashgate Farnham, UK 2009-12-00 99 112 Human Factors Issues in Combat Identification Most natural images have 1/fβ Fourier image statistics, a signature which is mimicked by fractals and which forms the basis for recent applications of fractals to camouflage. To distinguish a fractal camouflaged target (with 1/fβ* statistics) from a 1/fβ natural background (or another target), the exponents of target and background (or other target) must differ by a critical amount (dβ=β-β*), which varies depending on experimental circumstances. The same constraint applies for discriminating between friendly and enemy camouflaged targets. Here, we present data for discrimination of both static and dynamic fractal images, and data on how discrimination varies as a function of experimental methods and circumstances. The discrimination function has a minimum near β=1.6, which typifies images with less high spatial frequency content than the vast majority of natural images (β near 1.1). This implies that discrimination between fractal camouflaged objects is somewhat more difficult when the camouflaged objects are sufficiently similar in statistics to the statistics of natural images (as any sensible camouflage scheme should be), compared to the less natural β value of 1.6. This applies regardless of the β value of the background, which has implications for fratricide; friendlies and hostiles will be somewhat harder to tell apart for naturalistically camouflaged images, even when friendlies and hostiles are both visible against their backgrounds. The situation is even more perverse for “active camouflage”. Because of perceptual system nonlinearities (stochastic resonance), addition of dynamic noise to targets can actually enhance target detection and identification under some conditions. no notspecified http://www.kyb.tuebingen.mpg.de/ published 13 What visual discrimination of fractal textures can tell us about discrimination of camouflaged targets 15017 15422 SchwaningerWCC2006 2 A Schwaninger C Wallraven DW Cunningham SD Chiller-Glaus Elsevier Amsterdam, The Netherlands 2006-00-00 321–343 Understanding Emotions A deeper understanding of how the brain processes visual information can be obtained by comparing results from complementary fields such as psychophysics, physiology, and computer science. In this chapter, empirical findings are reviewed with regard to the proposed mechanisms and representations for processing identity and emotion in faces. Results from psychophysics clearly show that faces are processed by analyzing component information (eyes, nose, mouth, etc.) and their spatial relationship (configural information). Results from neuroscience indicate separate neural systems for recognition of identity and facial expression. Computer science offers a deeper understanding of the required algorithms and representations, and provides computational modeling of psychological and physiological accounts. An interdisciplinary approach taking these different perspectives into account provides a promising basis for better understanding and modeling of how the human brain processes visual information for recognition of identity and emotion in faces. no notspecified http://www.kyb.tuebingen.mpg.de/ published -321 Processing of facial identity and expression: a psychophysical, physiological, and computational perspective 15017 15422 1241 2 TF Shipley DW Cunningham Elsevier Amsterdam, The Netherlands 2001-00-00 557 585 From Fragments to Objects: Segmentation and Grouping in Vision no notspecified http://www.kyb.tuebingen.mpg.de/ published 28 Perception of occluding and occluded objects over time: Spatiotemporal segmentation and unit formation 15017 15422 634 46 DW Cunningham A Chatziastros M von der Heyde HH Bülthoff 2000-12-00 2000-12-00 Temporal adaptation and the role of temporal contiguity in spatial behavior no notspecified Temporal adaptation and the role of temporal contiguity in spatial behavior 15017 15422 1547 46 DW Cunningham VA Billock BH Tsou 2000-10-00 2000-10-00 Sensorimotor adaptation to violations of temporal contiguity no notspecified Sensorimotor adaptation to violations of temporal contiguity 15017 15422 AubreyCMRSW2012 7 AA Aubrey DW Cunningham D Marshall PL Rosin A Shin C Wallraven Cambridge, UK2012-05-22 22 2nd Joint AVA/BMVA Meeting on Biological and Machine Vision Facial expressions are one of the key modes of inter-personal communication for humans. Current research has almost exclusively focused on the so-called universal expressions (anger, disgust, fear, happy, sad, and surprise). Whereas these expressions are clearly important from an evolutionary point of view, their frequency of occurrence in daily life is rather low. We have recently begun investigating the processing of higher frequency, conversational expressions (e.g., agree, thinking, looking tired), with particular focus on the so-called backchannel expressions, that is, facial expressions a listener makes in reaction to a speaker. These expressions are believed to be crucial for controlling conversational flow. As there is no existing database of conversations, we recorded a large audio-visual corpus of conversations between pairs of people at Cardiff University. Two preliminary experiments have been conducted to empirically determine the sensitivity to changes in the backchannel. In the first experiment, eleven clips from several conversations were extracted. Each of the eleven main channels (“speaker”) was paired with four plausible and the real backchannel ("listener"). Participants were asked to choose the most appropriate backchannel. The second experiment examined sensitivity to temporal offsets in backchannel communication (one speaker-listener pair was shown at a time; the correct backchannel was always used, but its temporal onset was systematically varied). Results show that on average participants can correctly identify the correct backchannel sequence (41% correct; chance performance is 20%) and that people can tell if the back channel is early or late. Interestingly, it seems to be easier to judge lateness than earliness. In summary, the results conclusively show that -- despite the considerable difficulty of the task -- people are remarkably sensitive to the content and the timing of backchannel information. no notspecified http://www.kyb.tuebingen.mpg.de/ published -22 Sensitivity to backchannels in conversational expressions 15017 15422 6739 7 K Kaulard C Wallraven DW Cunningham HH Bülthoff Naples, FL, USA2010-05-00 606 10th Annual Meeting of the Vision Sciences Society (VSS 2010) Facial expressions form one of the most important and powerful communication systems of human social interaction. They express a large range of emotions but also convey more general, communicative signals. To date, research has mostly focused on the static, emotional aspect of facial expression processing, using only a limited set of “generic” or “universal” expression photographs, such as a happy or sad face. That facial expressions carry communicative aspects beyond emotion and that they transport meaning in the temporal domain, however, has so far been largely neglected. In order to enable a deeper understanding of facial expression processing with a focus on both emotional and communicative aspects of facial expressions in a dynamic context, it is essential to first construct a database that contains such material using a well-controlled setup. We here present the novel MPI facial expression database, which contains 20 native German participants performing 58 expressions based on pre-defined context scenarios, making it the most extensive database of its kind to date. Three experiments were performed to investigate the validity of the scenarios and the recognizability of the expressions. In Experiment 1, 10 participants were asked to freely name the facial expressions that would be elicited given the scenarios. The scenarios were effective: 82% of the answers matched the intended expressions. In Experiment 2, 10 participants had to identify 55 expression videos of 10 actors. We found that 34 expressions could be identified reliably without any context. Finally, in Experiment 3, 20 participants had to group the 55 expression videos of 10 actors based on similarity. Out of the 55 expressions, 45 formed consistent groups, which highlights the impressive variety of conversational expressions categories we use. Interestingly, none of the experiments found any advantage for the universal expressions, demonstrating the robustness with which we interpret conversational facial expressions. no notspecified http://www.kyb.tuebingen.mpg.de/ published -606 Laying the foundations for an in-depth investigation of the whole space of facial expressions 15017 15422 5954 7 K Kaulard C Wallraven DW Cunningham HH Bülthoff Regensburg, Germany2009-08-00 83 32nd European Conference on Visual Perception Investigations of facial expressions have focused almost exclusively on the six so-called universal expressions. During everyday interaction, however, a much larger set of facial expressions is used for communication. To examine this mostly unexplored space, we developed a large video database for emotional and conversational expressions: native German participants performed 58 expressions based on pre-defined context scenarios. Three experiments were performed to investigate the validity of the scenarios and the recognizability of the expressions. In Experiment 1, ten participants were asked to freely name the facial expressions that would be elicited given the scenarios. The scenarios were effective: 82% of the answers matched the intended expressions. In Experiment 2, ten participants had to identify 55 expression videos of ten actors, presented successively. We found that 20 expressions could be identified reliably without any context. Finally, in Experiment 3, twenty participants had to group the 55 expression videos based on similarity while allowing for repeated comparisons. Out of the 55 expressions, 45 formed a consistent group, respectively, showing that visual comparison facilitates the recognition of conversational expressions. Interestingly, none of the experiments found any advantage for the universal expressions, demonstrating the robustness with which we interpret conversational facial expressions. no notspecified http://www.kyb.tuebingen.mpg.de/ published -83 Going beyond universal expressions: investigating the visual perception of dynamic facial expressions 15017 15422 3875 7 J Fischer D Cunningham D Bartz C Wallraven HH Bülthoff W Strasser Tübingen, Germany2006-03-00 119 9th Tübingen Perception Conference (TWK 2006) In augmented reality, virtual graphical objects are overlaid over the real environment of the observer. Conventional augmented reality systems use standard computer graphics methods for generating the graphical representations of virtual objects. These renderings contain the typical artefacts of computer generated graphics, e.g., aliasing caused by the rasterization process and unrealistic, manually configured illumination models. Due to these artefacts, virtual objects look artificial and can easily be distinguished from the real environment. Recently, a different approach to generating augmented reality images was presented. In stylised augmented reality, similar types of artistic or illustrative stylisation are applied to the virtual objects and the camera image of the real environment [1]. Therefore, real and virtual image elements look more similar and are less distinguishable from each other. In this poster, we describe the results of a psychophysical study on the effectiveness of stylised augmented reality. A number of participants were asked to decide whether objects shown in images of augmented reality scenes are virtual or real. Conventionally rendered as well as stylised augmented reality images and short video clips were presented to the participants. The correctness of the participants&amp;amp;amp;#8217; responses and their reaction times were recorded. The results of our study clearly show that an equalized level of realism is achieved by using stylised augmented reality, i.e., that it is distinctly more difficult to discriminate virtual objects from real objects. no notspecified http://www.kyb.tuebingen.mpg.de/ published -119 Virtual or Real? Judging The Realism of Objects in Stylized Augmented Environments 15017 15422 3645 7 M Nusseck DW Cunningham C Wallraven HH Bülthoff A Coruña, Spain2005-08-00 204 28th European Conference on Visual Perception Facial expressions play a complex and important role in communication. A complete investigation of how facial expressions are recognised requires that different expressions be systematically and subtly manipulated. For this purpose, we recently developed a photo-realistic facial animation system that uses a combination of facial motion capture and high-resolution 3-D face scans. In order to determine if the synthetic expressions capture the subtlety of natural facial expressions, we directly compared recognition performance for video sequences of real-world and animated facial expressions (the sequences will be available on our website). Moreover, just as recognition of an incomplete or degraded object can be improved through prior experience with a complete, undistorted version of that object, it is possible that experience with the real-world video sequences may improve recognition of the synthesised expressions. Therefore, we explicitly investigated the effects of presentation order. More specifically, half of the participants saw all of the video sequences followed by the animation sequences, while the other half experienced the opposite order. Recognition of five expressions (agreement, disagreement, confusion, happiness, thinking) was measured with a six-alternative, non-forced-choice task. Overall, recognition performance was significantly higher ( p < 0.0001) for the video sequences (93%) than for the animations (73%). A closer look at the data showed that this difference is largely based on a single expression: confusion. As expected, there was an order effect for the animations ( p < 0.02): seeing the video sequences improved recognition performance for the animations. Finally, there was no order effect for the real videos ( p > 0.14). In conclusion, the synthesised expressions supported recognition performance similarly to real expressions and have proven to be a valuable tool in understanding the perception of facial expressions. no notspecified http://www.kyb.tuebingen.mpg.de/ published -204 Perceptual validation of facial animation: The role of prior experience 15017 15422 2535 7 T Cooke DW Cunningham C Wallraven Tübingen, Germany2004-02-00 65 7th Tübingen Perception Conference (TWK 2004) Patterns of abrupt changes in a scene, such as the dynamic occlusion of texture elements (causing their apppearance and disappearance), can give rise to the perception of the edges of the occluder via a process called Spatiotemporal Boundary Formation (SBF). It has previously been shown that SBF can be disrupted by very small amounts of dynamic noise spread globally throughout a scene. We recently developed a mathematical model of SBF in which groups of local changes are used to extract edges, which are then combined into a gure and used to estimate the gure's motion. The model implies that SBF relies on local processing and predicts that SBF should be impaired when noise is added near the edges of the gure, but not when it is added far from the edges. This prediction was tested in a shape-identication task in which the location of noise is varied. Indeed, performance was not impaired by noise far from the gure, but was markedly disrupted by noise near the gure, supporting the notion that changes are integrated locally rather than globally during SBF. In the second part of this project, the mathematical model of SBF was implemented in software. Reichardt-based motion detectors were used to lter the experimental stimuli and provide the input to the software implementation. Three simple geometrical gures, similar to those used in the psychophysical experiment, were reconstructed using this method, demonstrating one way in which a mid-level visual mechanism such as SBF could connect low-level mechanisms such as change detection to higher-level mechanisms such as shape detection. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf2535.pdf published -65 Local Processing in Spatiotemporal Boundary Formation 15017 15422 MunkaKC2003 7 L Munka C Kaernbach DW Cunningham Tübingen, Germany2003-02-00 128 6. Tübinger Wahrnehmungskonferenz (TWK 2003) In order to pick up an object, its visual location must be converted into the appropriate motor commands. Introducing a discrepancy between the seen and felt locations of the object (e.g., via prism goggles) initially impairs the ability to touch it. The sensory system rapidly adapts to the discrepancy, however, returning perception and performance to near normal. Subsequent removal of the discrepancy leads to a renewed performance decrement - a negative aftere ect (NAE). It is generally believed that the process of adaptation consists primarily of \recalibrating" the transformation between the visual and proprioceptive perception of spatial location (Bedford, The psychology of learning and motivation, 1999). According to such a purely perceptual account of adaptation, the movement to reach the object is not important. If, however, the transformation from perception to action is altered, then it will be dependent on motion - i.e. changing motion parameters will reduce or eliminate the NAE (see also Martin et al., Brain, 1996). According to our hypothesis spatial visuomotor information is distributively stored and changed by prism adaptation and it is not based on a centrally organized spatial information system. We conducted seven experiments consisting of four blocks each, in which participants had to touch a cross presented at eye level on a touch screen. In the rst block the participants were introduced and familiarized with the experiment. Blocks two and four were pre and post tests to measure the NAE produced during the di erent experimental conditions in block 3 in which the participants were wearing prism goggles: we tested the e ects of di erent trajectories, di erent starting points, weight, vertical generalization and di erent types of feedback. A total transfer from an adapted to a non-adapted condition didn't turn up in any of our experiments, although the trajectories where highly identical in some of them. It rather seems that newly learned spatial information in prism adaptation experiments is stored and retrieved distributively for di erent extremities, for di erent trajectories and for di erent stress/strain conditions (e.g. weight). Furthermore, transfer seems to become weaker with bigger di erences in location. Therefore we conclude that no visual \recalibration" is taking place but a relearning of distributetively organized parameters of visuomotor coordination. no notspecified http://www.kyb.tuebingen.mpg.de/ published -128 Visuomotor Adaptation: Dependency on Motion Trajectory 15017 15422 2014 7 A Schwaninger DW Cunningham M Kleiner Kansas City, KS, USA2002-11-21 10th Annual Workshop on Object Perception and Memory (OPAM 2002) Inverting the eyes and the mouth within the facial context produces a bizarre facial expression when the face is presented upright but not when the face is inverted (Thatcher illusion, Thompson, 1980). In the present study we investigated whether this illusion is part-based or holistic and whether motion increases bizarreness. Static upright Thatcher faces were rated more bizarre than the eyes and mouth presented in isolation suggesting an important role of context and holistic processing. As expected, inverted facial stimuli were perceived much less bizarre. Interestingly, isolated parts were more bizarre than the whole thatcherized face when inverted. Adding motion to the smiling thatcherized faces increased bizarreness in all conditions (parts vs. whole, upright vs. inverted). These results were replicated in a separate experiment with talking instead of smiling faces and are discussed within an integrative model of face processing. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 Moving the Thatcher Illusion 15017 15422 1771 7 DW Cunningham ABA Graf HH Bülthoff Sarasota, FL, USA2002-11-00 704 Second Annual Meeting of the Vision Sciences Society (VSS 2002) When a camouflaged animal sits in front of the appropriate background, the animal is effectively invisible. As soon as the animal moves, however, it is easily visible despite the fact that there is still no static shape information. Its shape is perceived solely by the pattern of changes over time. This process, referred to as Spatiotemporal Boundary Formation (SBF), can be initiated by a wide range of texture transformations, including changes in the visibility, shape, or color of individual texture elements. Shipley and colleagues have gathered a wealth of psychophysical data on SBF, and have presented a mathematical proof of how the orientation of local edge segments (LESs) can be recovered from as few as 3 element changes (Shipley and Kellman, 1997). Here, we extend this proof to the extraction of global form and motion. More specifically, we present a model that recovers the orientation of the LESs from a dataset consisting of the relative spatiotemporal location of the element changes. The recovered orientations of as few as 2 LESs can then be used to extract the global motion, which is then used to determine the relative spatiotemporal location and minimal length of the LESs. Computational simulations show that the model captures the major psychophysical aspects of SBF, including a dependency on the spatiotemporal density of element changes, a sensitivity to spurious changes, an ability to extract more than one figure at a time, and a tolerance for a non-constant global motion. Unlike Shipley and Kellman's earlier proof, which required that pairs of element changes be represented as local motion vectors, the present model merely encodes the relative spatiotemporal locations of the changes. This usage of a relative encoding scheme yields several emergent properties that are strikingly similar to the perception of aperture viewed figures (Anorthoscopic perception). This offering the possibility of unifying the two phenomena within a single mathematical model. no notspecified http://www.kyb.tuebingen.mpg.de/ published -704 A relative encoding approach to modeling Spatiotemporal Boundary Formation 15017 15422 2237 7 DW Cunningham IM Thornton NF Troje HH Bülthoff Glasgow, UK2002-08-00 120 25th European Conference on Visual Perception Biological motion contains many forms of information. Observers are usually able to tell 'what' action is being performed (eg walking versus running), 'how' it is being performed (eg quickly versus slowly), and by 'whom' (eg a young versus an old actor). We used visual search to explore the perception of gender-from-motion. In the first experiment, we used computer-animated, fully rendered human figures in which the structural and dynamic information for gender were factorially combined. In separate blocks, observers were asked to locate a figure walking with a male or female gait among distractors having the same form but opposite motion. In the second experiment, point-light walkers moved along random paths in a 3-D virtual environment. Observers were asked to locate a figure walking with a male or female gait among distractors with the opposite motion. In both experiments, the set size was varied between 1 and 4, and targets were present on 50% of the trials. The results suggest that (a) visual search can be used to explore gender-from-motion, (b) extraction of gender-from-motion is fairly inefficient (search slopes often exceed 500 ms item-1), and (c) there appears to be an observer-gender by target - gender interaction, with male observers producing lower RTs for female targets and vice versa. no notspecified http://www.kyb.tuebingen.mpg.de/ published -120 Searching for gender-from-motion 15017 15422 1449 7 C Kaernbach DW Cunningham Chemnitz, Germany2002-03-00 139 44. Tagung Experimentell Arbeitender Psychologen (TeaP 2002) Ich sitze vor meinem Schreibtisch und sehe auf ihm einen Stift. Ich möchte ihn in die Hand nehmen. Dazu müßte ich wissen, wo er ist. "Du siehst doch, wo er ist." So einfach ist das nicht. Mir liegt ein verzerrtes Netzhautbild vor, das sich ständig ändert, während mein Blick über den Schreibtisch wandert. Ich muß Blickrichtung, Kopfstellung und Körperhaltung berücksichtigen, um eine erfolgreiche Greifhandlung durchzuführen. Das gelingt uns so natürlich, daß wir die Schwierigkeit der Aufgabe leicht unterschätzen. Ein klassischer Untersuchungsansatz zum Studium der räumlichen Repräsentation ist das Adaptieren zu optischen Transformationen. Wir haben eine Prismenbrille verwendet, die das Bild seitlich um 19° verschiebt. Um die Frage nach einer zentralen versus einer distribuierten räumlichen Repräsentation zu klären, wurde dabei eine bestimmte Bewegung geübt, und dann der Transfer auf andere Bewegungen gemessen. Die Ergebnisse sprechen dafür, daß räumliches Wissen nicht nur auf die einzelnen motorischen Systeme, sondern sogar auf verschiedene Bewegungstrajektorien verteilt ist. no notspecified http://www.kyb.tuebingen.mpg.de/ published -139 Visuell-motorische Adaption unter optischen Transformationen 1007 7 DW Cunningham ABA Graf HH Bülthoff Tübingen, Germany2002-02-00 77 5. Tübinger Wahrnehmungskonferenz (TWK 2002) When a camouflaged animal sits in front of the appropriate background, the animal is effectively invisible. As soon as the animal moves, however, it is easily visible despite the fact that at any given instant, there is no shape information. This process, referred to as Spatiotemporal Boundary Formation (SBF), can be initiated by a wide range of texture transformations, including changes in the visibility, shape, or color of individual texture elements. Shipley and colleagues have gathered a wealth of psychophysical data on SBF, and have presented a local motion vector model for the recovery of the orientation of local edge segments (LESs) from as few as three element changes (Shipley and Kellman, 1997). Here, we improve and extend this model to cover the extraction of global form and motion. The model recovers the orientation of the LESs from a dataset consisting of the relative spatiotemporal location of the element changes. The recovered orientations of as few as two LESs is then be used to extract the global motion, which is then used to determine the relative spatiotemporal location and minimal length of the LESs. To complete the global form, the LESs are connected in a manner similar to that used in illusory contours. Unlike Shipley and Kellman’s earlier model, which required that pairs of element changes be represented as local motion vectors, the present model merely encodes the relative spatiotemporal locations of the changes in any arbitrary coordinate system. Computational simulations of the model show that it captures the major psychophysical aspects of SBF, including a dependency on the spatiotemporal density of element changes and a sensitivity to spurious changes. Interestingly, the relative encoding scheme yields several emergent properties that are strikingly similar to the perception of aperture viewed figures (Anorthoscopic Perception). The model captures many of the important qaulities of SBF, and offers a framework within which additional aspects of SBF may be modelled. Moreover, the relative encoding approach seems to inherently encapsulate other phenomenon, offering the possibility of unifying several phenomena within a single mathematical model. no notspecified http://www.kyb.tuebingen.mpg.de/ published -77 A relative encoding model of spatiotemporal boundary formation 15017 15422 1388 7 L Munka C Kaernbach DW Cunningham Tübingen, Germany2002-02-00 142 5. Tübinger Wahrnehmungskonferenz (TWK 2002) In order to pick up an object, its visual location must be converted into the appropriate motor commands. Introducing a discrepancy between the seen and felt location of the object (e.g., via prism goggles) initially impairs our ability to touch it. The sensory systems rapidly adapt to the discrepancy, however, returning perception and performance to near normal. Subsequent removal of the discrepancy leads to a renewed performance decrement -- a Negative Aftereffect (NAE). It is generally believed that this adaptation consists primarily of “recalibrating” the transformation between the visual and proprioceptive perception of spatial location (Bedford, 1999). According to such a purely perceptual account of adaptation, the exact path used to reach the object should not be important. If, however, it is the transformation from perception to action that is being altered, then changing the motion trajectory should reduce or eliminate the NAE. Starting with both hands on the desktop, the chin resting on a horizontal bar, participants (N=72) had to touch a cross presented at eye level on a touch screen 30 cm in front of them. Four trajectories were possible: reaching to the cross from below or (swinging the arm backwards) from above the bar, using either their left or their right hand. Reaching Accuracy without feedback was determined for all four trajectories before and after adaptation to a single trajectory with prism goggles (19° horizontal displacement). The NAE was 46mm (8.7°) for the adapted trajectory, 26mm negligable for both trajectories of the other hand. The NAE was larger for unfamiliar (above bar, or usage of non-preferred hand) than for familiar trajectories. Visuomotor adaptation is not merely a perceptual recalibration. Not only does the structure of the motion trajectory play a central role, but the familiarity of the trajectory also seems to be important. These results have strong implications for all models of visuomotor adaptation. no notspecified http://www.kyb.tuebingen.mpg.de/ published -142 Prism adaptation: Dependency on motion trajectory 15017 15422 1410 7 IM Thornton DW Cunningham NF Troje HH Bülthoff Sarasota, FL, USA2001-12-00 354 First Annual Meeting of the Vision Sciences Society (VSS 2001) Johansson's (1973) point light walkers remain one of the most compelling demonstrations of how motion can determine the perception of form. Most studies of biological motion perception have presented isolated point-light figures in unstructured environments. Recently we have begun to explore the perception of human motion using more naturalistic displays and tasks. Here, we report new findings on the perception of gender using a visual search paradigm. Three-dimensional walking sequences were captured from human actors (4 male, 4 female) using a 7 camera motion capture system. These sequences were processed to produce point-light computer models which were displayed in a simple virtual environment. The figures appeared in a random location and walked on a random path within the bounds of an invisible virtual arena. Walkers could face and move in all directions, moving in both the frontal parallel plane and in depth. In separate blocks observers searched for a male or a female target among distractors of the opposite gender. Set size was varied from between 1 and 4. Targets were present on 50% of trials. Preliminary results suggest that both male and female observers can locate targets of the opposite gender faster than targets of the same gender. no notspecified http://www.kyb.tuebingen.mpg.de/ published -354 “You can tell by the way I use my walk …”: New studies of gender and gait 15017 15422 640 7 DW Cunningham BW Kreher M von der Heyde HH Bülthoff Sarasota, FL, USA2001-12-00 135 First Annual Meeting of the Vision Sciences Society (VSS 2001) Delaying the presentation of information to one modality relative to another (an intersensory temporal offset) impairs performance on a wide range of tasks. We have recently shown, however, that a few minutes exposure to delayed visual feedback induces sensorimotor temporal adaptation, returning performance to normal. Here, we examine whether adaptation to delayed vestibular feedback is possible. Subjects were placed on a motion platform, and were asked to perform a stabilization task. The task was similar to balancing a rod on the tip of your finger. Specifically, the platform acted as if it were on the end of an inverted pendulum, with subjects applying an acceleration to the platform via a joystick. The more difficulty one has in stabilizing the platform the more it will oscillate, increasing the variability in the platform's position. The experiment was divided into 3 sections. During the Baseline section (5 minutes), subjects performed the task with immediate vestibular feedback. They then were presented with a Training section, consisting of 4 sessions (5 minutes each) during which vestibular feedback was delayed by 500 ms. Finally, subjects were presented with a Post-test (two minutes) with no feedback delay. Subjects performed rather well in the Baseline section (average standard deviation of platform tilt was 1.37 degrees). The introduction of the delay greatly impaired performance (8.81 degrees standard deviation in the 1st Training session), but performance rapidly showed significant improvement (5.59 degrees standard deviation during the last training section, p<0.04). Subjects clearly learned to compensate, at least partially, for the delayed vestibular feedback. Performance during the Post-test was worse than during Baseline (2.48 degrees standard deviation in tilt). This decrease suggests that the improvement seen during training might be the result of intersensory temporal adaptation. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf640.pdf published -135 Do cause and effect need to be temporally continuous? Learning to compensate for delayed vestibular feedback 15017 15422 1012 7 WO Readinger A Chatziastros DW Cunningham JE Cutting HH Bülthoff Sarasota, FL, USA2001-12-00 136 First Annual Meeting of the Vision Sciences Society (VSS 2001) A large portion of current and previous research on locomotor and vehicle navigation tends to assume that people choose a goal or destination in the visual world and then generally look where they are going. There exists, however, considerable anecdotal evidence and observational data suggesting that humans also will tend to go where they are looking. Considering the amount of time a pedestrian or driver spends looking away from his precise heading point, this tendency has received relatively little experimental attention. The goal of the present research is to determine how gaze eccentricity affects drivers' abilities to steer a straight course. A high-performance virtual reality theater was used to simulate the motion of a car through a textured environment with the participant controlling direction of travel via a forced-feedback steering wheel. Participants (n=12) were asked to fixate a Landolt-C figure in one of 7 positions (0, +/- 15, 30, or 45 degrees from center) and drive down the center of a perfectly straight road. The Landolt-C was located just above the horizon in a fixed position on the viewing screen. Throughout each trial, the orientation of the figure varied randomly between 4 possible positions (0, 90, 180, and 270 degrees) and, in order to ensure fixation, subjects were required to respond to particular orientations. Lateral position of the driver was recorded for each of the different gaze eccentricities. Significant deviations from straight ahead were found for side of presentation when compared to fixation at 0 degrees (p<0.01). That is, when participants fixated to the left, for example, they systematically steered in the same direction. These results are compared to another study in which subjects' performance was measured while their head movements were restricted using a head-strap and chin-rest. The similar pattern of results in both conditions will be discussed in terms of the influence of retinal flow on the control of locomotion. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf1012.pdf published -136 Gaze-eccentricity effects on automobile driving performance, or going where you look 15017 15422 630 7 M von der Heyde BE Riecke DW Cunningham HH Bülthoff Sarasota, FL, USA2001-12-00 188 First Annual Meeting of the Vision Sciences Society (VSS 2001) In most virtual reality (VR) applications turns are misperceived, which leads to disorientation. Here we focus on two cues providing no absolute spatial reference: optic flow and vestibular cues. We asked whether: (a) both visual and vestibular information are stored and can be reproduced later; and (b) if those modalities are integrated into one coherent percept or if the memory is modality specific. We used a VR setup including a motion simulator (Stewart platform) and a head-mounted display for presenting vestibular and visual stimuli, respectively. Subjects followed an invisible randomly generated path including heading changes between 8.5 and 17 degrees. Heading deviations from this path were presented as vestibular roll rotation. Hence the path was solely defined by vestibular (and proprioceptive) information. The subjects' task was to continuously adjust the roll axis of the platform to level position. They controlled their heading with a joystick and thereby maintained an upright position. After successfully following a vestibularly defined path twice, subjects were asked to reproduce it from memory. During the reproduction phase, the gain between the joystick control and the resulting visual and vestibular turns were independently varied. Subjects learned and memorized curves of the vestibularly defined virtual path and were able to reproduce the amplitudes of the turns. This demonstrates that vestibular signals can be used for spatial orientation in virtual reality. Since the modality with the bigger gain factor had a dominant effect on the reproduced turns, the integration of visual and vestibular information seems to follow a “max rule”, in which the larger signal is responsible for the perceived and memorized heading change. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf630.pdf published -188 No visual dominance for remembered turns - Psychophysical experiments on the integration of visual and vestibular cues in Virtual Reality 15017 15422 655 7 WO Readinger A Chatziastros DW Cunningham HH Bülthoff Kuşadasi, Turkey2001-08-00 109 Twenty-fourth European Conference on Visual Perception There has been growing evidence expressing the computational (and perhaps practical) difficulty of recovering heading from retinal flow when the observer is looking away from his path. Despite this, it is generally accepted that retinal-flow information plays a significant role in the control of locomotion. The experiments reported here attempt to address one meaningful behavioural consequence associated with this situation. Specifically, we consider eccentric gaze and its effects on automobile steering. In three conditions, we measured drivers' steering performance on a straight road, located in a textured ground plane, and presented in a 180 deg field-of-view projection theatre. Consistent with earlier findings, at eccentricities from 15 to 45 deg away from heading direction, subjects' lateral position on the road tended significantly towards their direction of gaze (p < 0.001), but eccentricities of as little as 5 deg from heading direction also significantly affected position on the road surface (p < 0.01). Furthermore, this effect was found to scale based on small (±5 deg) changes in gaze-movement angle, but not with speed of travel through the environment. We propose, therefore, a model of steering performance in such situations resulting from characteristics of the retinal flow immediately surrounding the point of fixation. no notspecified http://www.kyb.tuebingen.mpg.de/ published -109 Driving effects of retinal flow properties associated with eccentric gaze 15017 15422 1246 7 DW Cunningham HH Bülthoff Kuşadasi, Turkey2001-08-00 102 Twenty-fourth European Conference on Visual Perception The consequences of an action almost always occur immediately. Delaying the consequences of an action (eg by delaying visual feedback) drastically impairs performance on a wide range of tasks. A few minutes of exposure to a delay can, however, induce sensorimotor temporal adaptation. Here we ask whether a stable delay is necessary for temporal adaptation. Specifically, we examined performance in a driving simulator (where subjects could control the direction but not the speed of travel). The delay was on average 250 ms, but fluctuated rapidly (36 Hz) and randomly between 50 and 450 ms. Overall, subjects were able to learn to drive a virtual car with a variable delay. In one experiment, we found that the adapted state also improved performance on untrained streets (generalisation). In a second experiment, performance with immediate feedback was measured both before and after delay training. We found a strong negative aftereffect (approximately 50% drop in performance from pre- to post-test). While some behavioural strategies (eg slow gradual changes in steering wheel angle) might mitigate the impact of a variable delay, these strategies do not totally eliminate the variability, particular for fast speeds and sharp corners. no notspecified http://www.kyb.tuebingen.mpg.de/ published -102 Temporal Adaptation with a variable delay 15017 15422 67 7 WO Readinger A Chatziastros DW Cunningham JE Cutting HH Bülthoff Tübingen, Germany2001-03-00 149 4. Tübinger Wahrnehmungskonferenz (TWK 2001) Applied navigation tasks, such as driving a car, present unique opportunities to study the human perception/action system. Traditionally, research into the control of locomotion has assumed that humans choose a destination and then generally look where they go. However, evidence from motorcycle and equitation manuals, for instance, suggests that the reciprocal behavior is also important. That is, even with a distinct goal in the environment, people tend to navigate in the direction they are looking, only occasionally checking on their progress toward a destination and making adjustments as necessary. Considering the implications for the performance and safety of drivers, the present study is designed to investigate effects of gaze-eccentricity on basic steering abilities. Using a 180-degree horizontal FoV projection theater, we simulated a car moving through a textured environment while participants used a forced-feedback steering-wheel to control direction of travel. During each trial, participants (n=12) were asked to fixate a Landolt-C figure which was displayed in one of 7 positions (0, +/- 15, 30, or 45 degrees from center) anchored on the screen, and drive down the center of a straight road. In order to ensure fixation, the orientation of the Landolt-C varied randomly between 4 positions (0, 90, 180, and 270 degrees) and participants were required to respond to particular orientations by pressing a button on the steering-wheel. The lateral position of the driver was measured during the trial. In this basic condition, significant deviations from straight ahead were found when conditions of eccentric gaze were compared to fixation at 0 degrees (p<0.001). Specifically, fixation to one side of the street systematically lead the driver to steer in that direction. These results are similar to the findings from another condition in which participants' head movements were restricted. The contribution of retinal flow to this pattern of data will be discussed, along with reports of driver experience and confidence. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf67.pdf published -149 Gaze-direction effects on drivers' abilities to steer a straight course 15017 15422 59 7 DW Cunningham BW Kreher M von der Heyde HH Bülthoff Tübingen, Germany2001-03-00 151 4. Tübinger Wahrnehmungskonferenz (TWK 2001) In order to rapdily and accurately interact with the world, we need to perceive the consequences of our actions. It should not be surprising, then, that delaying the consequences of our actions, or delaying feedback about our actions, impairs performance on a wide range of tasks. We have recently shown that a few minutes exposure to delayed visual feedback induces sensorimotor temporal adaptation, returning performance to near normal levels. While visual feedback plays a large role in many tasks, there are some tasks for which vestibular perception is more critical. Here, we examine whether adaptation to delayed vestibular feedback is possible. To test for vestibular temporal adaptation, subjects were placed on a motion platform and were asked to perform a stabilization task. The task was similar to balancing a rod on the tip of your finger. Specifically, the platform acted as if it were on the end of an inverted pendulum. Subjects moved the platform by applying an acceleration to it via a joystick. The experiment was divided into 3 sections. During the Baseline section, which lasted 5 minutes, subjects performed the task with immediate vestibular feedback. They then were presented with a Training section, which consisted of 4 sessions (5 minutes each) during which vestibular feedback was delayed by 500 ms. Finally, subjects performance on the task with immediate feedback was remeasured during a 2 minute Post-test. The more difficulty one has in stabilizing the platform the more it will oscillate, increasing the variablilty in the platform's position and orientation. Accordingly, positional variance served as the primary measure of the subjects' performance. Subjects did rather well in the Baseline section (average standard deviation of platform tilt was 1.37 degrees). The introduction of the delay greatly impaired performance (8.81 degrees standard deviation in the 1st Training session), but performance rapidly showed significant improvement (5.59 degrees standard deviation during the last training session). Subjects clearly learned to compensate, at least partially, for the delayed vestibular feedback. Performance during the Post-test showed a negative aftereffect: The performance with a 500 ms delay worse during the Post-test than during Baseline (2.48 degrees versus 1.37 degreees), suggesting that the improvement seen during training was the result of intersensory temporal adaptation. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf59.pdf published -151 Temporal adaptation to delayed vestibular feedback 15017 15422 63 7 M von der Heyde BE Riecke DW Cunningham HH Bülthoff Tübingen, Germany2001-03-00 142 4. Tübinger Wahrnehmungskonferenz (TWK 2001) Perception of ego turns is crucial for navigation and self-localization. Yet in most virtual reality (VR) applications turns are misperceived, which leads to disorientation. Here we focus on two cues providing no absolute spatial reference: optic flow and vestibular cues. We asked whether: (a) both visual and vestibular information are stored and can be reproduced later; and (b) if those modalities are integrated into one coherent percept or if the memory is modality specific. In the following experiment, subjects learned and memorized turns and were able to reproduce them even with different gain factors for the vestibular and visual feedback. We used a VR setup including a motion simulator (Stewart platform) and a head-mounted display for presenting vestibular and visual stimuli, respectively. Subjects followed an invisible randomly generated path including heading changes between 8.5 and 17 degrees. Heading deviations from this path were presented as vestibular roll rotation. Hence the path was solely defined by vestibular (and proprioceptive) information. One group of subjects' continuously adjusted the roll axis of the platform to level position. They controlled their heading with a joystick and thereby maintained an upright position. The other group was passively guided through the sequence of heading turns without any roll signal. After successfully following a vestibularly defined path twice, subjects were asked to reproduce it from memory. During the reproduction phase, the gain between the joystick control and the resulting visual and vestibular turns were independently varied by a factor of 1/sqrt(2), 1 or sqrt(2). Subjects from both groups learned and memorized curves of the vestibularly defined virtual path and were able to reproduce the amplitudes of the turns. This demonstrates that vestibular signals can be used for spatial orientation in virtual reality. Since the modality with the bigger gain factor had for both groups a dominant effect on the reproduced turns, the integration of visual and vestibular information seems to follow a "max rule", in which the larger signal is responsible for the perceived and memorized heading change. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf63.pdf published -142 Visual-vestibular sensor integration follows a max-rule: results from psychophysical experiments in virtual reality 15017 15422 1995 7 DW Cunningham VA Billock BH Tsou Groningen, Netherlands2000-08-00 104 23rd European Conference on Visual Perception (ECVP 2000) Natural scenes are fractal in space (ie they have 1/f B spatial frequency spectra) and time (1/f A temporal spectra), and can be compellingly mimicked by fractal textures. If dynamic fractal texture statistics are used to describe natural scenes, then data on discriminability of such textures are required. The smallest detectable change was measured separately for 10 spatial (0.4 to 2.2) and 8 temporal exponents (static, and 0.2 to 1.4) with an adaptive staircase. Computational constraints limited each fractal to 64 frames (~ 2 s) of 64 × 64 pixel images. Spatial discriminations were easiest when the spatial exponent B was ~ 1.6 and were similar across all temporal exponents. Temporal discriminations were easiest when the temporal exponent A was ~ 0.8, and increased in difficulty as the spatial exponent increased. This similarity in spatial discrimination thresholds for static and dynamic fractals suggests that the spatial and temporal dimensions are independent in dynamic fractals (at least for spatial judgments), as is often assumed. The dependence of temporal judgments on the coarseness of the texture (ie on the spatial exponent) is understandable, as a 1 mm change in position is more noticeable for a 1 mm object than for a 100 m object. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf1995.pdf published -104 Spatiotemporal discrimination thresholds for dynamic random fractal (1/f) textures 15017 15422 114 7 DW Cunningham M von der Heyde HH Bülthoff Fort Lauderdale, FL, USA2000-05-00 S246 Annual Meeting of the Association for Research in Vision and Ophthalmology (ARVO 2000) no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf114.pdf published 0 Learning to drive with delayed visual feedback 15017 15422 164 7 M von der Heyde BE Riecke DW Cunningham HH Bülthoff San Francisco, CA, USA2000-04-00 77 7th Annual Meeting of the Cognitive Neuroscience Society Purpose: The vestibular system is known to measure accelerations for linear forward movements. Can humans integrate these vestibular signals to derive reliably distance and velocity estimates? Methods: Blindfolded naive volunteers participated in a psychophysical experiment using a Stewart-Platform motion simulator. The vestibular stimuli consisted of Gaussian-shaped translatory velocity profiles with a duration of less than 4 seconds. The full two-factorial design covered 6 peak accelerations above threshold and 5 distances up to 25cm with 4 repetitions. In three separate blocks, the subjects were asked to verbally judge on a scale from 1 to 100 traveled distance, maximum velocity and maximum acceleration. Results: Subjects perceived distance, velocity and acceleration quite consistently, but with systematic errors. The distance estimates showed a linear scaling towards the mean and were independent of accelerations. The correlation of perceived and real velocity was linear and showed no systematic influence of distances or accelerations. High accelerations were drastically underestimated and accelerations close to threshold were overestimated, showing a logarithmic dependency. Conclusions: Despite the fact that the vestibular system measures acceleration only, one can derive peak velocity and traveled distance from it. Interestingly, even though maximum acceleration was perceived non linear, velocity and distance was judged consistently linear. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf164.pdf published -77 Humans can extract distance and velocity from vestibular perceived acceleration 15017 15422 113 7 DW Cunningham M von der Heyde HH Bülthoff Tübingen, Germany2000-02-00 164 3. Tübinger Wahrnehmungskonferenz (TWK 2000) The consequences of an action usually occur immediately. One of the more important ramifications of this is that delaying visual feedback greatly impairs performance on a wide range of tasks. Cunningham et al. (ARVO 1999) have demonstrated that with practice, humans can perform equally well with delayed and immediate visual feedback on a simple obstacle avoidance task with abstract stimuli. Here, we examine the effects of training in more detail under more realistic conditions. Naive volunteers maneuvered a virtual car along a convoluted path in a high-fidelity virtual environment, which was projected onto a 180 deg. screen. Subjects drove at a constant speed, steering with a forced-feedback steering wheel. In Exp. 1, subjects were presented with 7 speeds in random order 5 times, using immediate visual feedback and a single path. Subsequently, subjects trained with a 280 ms delay, and then were presented with 5 trials at the fastest speed they had successfully completed in the first section. In Exp. 2, subjects were given 15 trials of practice using immediate feedback. Following this, subjects’ performance with 5 paths at 3 speeds was measured, then they trained on a new path, and finally they were presented with 5 new paths at the 3 speeds. In both experiments, training with delayed feedback improved performance accuracy with delayed feedback, and seemed to reduce the perceptual magnitude of the delay. In Exp. 1, the training also lowered performance with immediate feedback. In Exp. 2, the improved performance generalized to novel paths. These results are the main hallmarks for sensorimotor adaptation, and suggest that humans can adapt to intersensory temporal differences. Regardless of the underlying mechanism, however, it is clear that accurate control of vehicles at high speeds with delayed feedback can be learned. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf113.pdf published -164 Driving a virtual car with delayed visual feedback 15017 15422 165 7 M von der Heyde BE Riecke DW Cunningham HH Bülthoff Tübingen, Germany2000-02-00 148 3. Tübinger Wahrnehmungskonferenz (TWK 2000) The vestibular system is known to measure changes in linear and angular position changes in terms of acceleration. Can humans judge these vestibular signals as acceleration and integrate them to reliably derive distance and velocity estimates? Twelve blindfolded naive volunteers participated in a psychophysical experiment using a Stewart-Platform motion simulator. The vestibular stimuli consisted of Gaussian-shaped translatory or rotatory velocity profiles with a duration of less than 4 seconds. The full two-factorial design covered 6 peak accelerations above threshold and 5 distances with 4 repetitions. In three separate blocks, the subjects were asked to verbally judge on a scale from 1 to 100 the distance traveled or the angle turned, maximum velocity and maximum acceleration. Subjects judged the distance, velocity and acceleration quite consistently, but with systematic errors. The distance estimates showed a linear scaling towards the mean response and were independent of accelerations. The correlation of perceived and real velocity was linear and showed no systematic influence of distances or accelerations. High accelerations were drastically underestimated and accelerations close to threshold were overestimated, showing a logarithmic dependency. Therefore, the judged acceleration was close to the velocity judgment. There was no significant difference between translational and angular movements. Despite the fact that the vestibular system measures acceleration only, one can derive peak velocity and traveled distance from it. Interestingly, even though maximum acceleration was perceived non-linearly, velocity and distance judgments were linear. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf165.pdf published -148 Humans can separately perceive distance, velocity and acceleration from vestibular stimulation 15017 15422 1247 7 DW Cunningham BH Tsou Fort Lauderdale, FL, USA1999-05-00 585 Annual Meeting of the Association for Research in Vision and Ophthalmology (ARVO 1999) no notspecified http://www.kyb.tuebingen.mpg.de/ published -585 Sensorimotor adaptation to temporally displaced feedback 1248 7 DW Cunningham PR Havig JS Chen VA Billock BH Tsou Fort Lauderdale, FL, USA1998-05-00 859 Annual Meeting of the Association for Research in Vision and Ophthalmology (ARVO 1998) no notspecified http://www.kyb.tuebingen.mpg.de/ published -859 Perception of spatiotemporal random fractal textures: Towards a colorimetry of dynamic texture 1249 7 DW Cunningham TF Shipley PJ Kellman 1997-00-00 S1005 Investigative Opthalmoology and Visual Science Supplement no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 The roles of spatial and spatiotemporal surface information in Spatiotemporal Boundary Formation. 1250 7 DW Cunningham TF Shipley PJ Kellman Fort Lauderdale, FL, USA1996-04-00 172 Annual Meeting of the Association for Research in Vision and Ophthalmology (ARVO 1996) no notspecified http://www.kyb.tuebingen.mpg.de/ published -172 Spatiotemporal Boundary Formation: The role of global motion signals 5751 14 DW Cunningham 2007-00-00 no notspecified published Perceptual Graphics habilitation KaulardWCB2009 10 K Kaulard C Wallraven DW Cunningham HH Bülthoff 5920 10 D Cunningham C Wallraven 4595 10 M Nusseck DW Cunningham C Wallraven HH Bülthoff 2319 10 M Breidt C Wallraven DW Cunningham HH Bülthoff 652 10 A Chatziastros DW Cunningham HH Bülthoff