SchultzBBP2012 3 J Schultz M Brockhaus HH Bülthoff K Pilz 2013-05-00 5 23 1167 1178 Cerebral Cortex Facial motion carries essential information about other people's emotions and intentions. Most previous studies have suggested that facial motion is mainly processed in the superior temporal sulcus (STS), but several recent studies have also shown involvement of ventral temporal face-sensitive regions. Up to now, it is not known whether the increased response to facial motion is due to an increased amount of static information in the stimulus, to the deformation of the face over time, or to increased attentional demands. We presented nonrigidly moving faces and control stimuli to participants performing a demanding task unrelated to the face stimuli. We manipulated the amount of static information by using movies with different frame rates. The fluidity of the motion was manipulated by presenting movies with frames either in the order in which they were recorded or in scrambled order. Results confirm higher activation for moving compared with static faces in STS and under certain conditions in ventral temporal face-sensitive regions. Activation was maximal at a frame rate of 12.5 Hz and smaller for scrambled movies. These results indicate that both the amount of static information and the fluid facial motion per se are important factors for the processing of dynamic faces. no notspecified http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/2012/Cerebral-Cortex-2012-Schultz.pdf published 11 15017 15422 HelbigERPTMSN2011 3 HB Helbig MO Ernst E Ricciardi P Pietrini A Thielscher KM Mayer J Schultz U Noppeney 2012-04-00 2 60 1063–1072 NeuroImage Behaviourally, humans have been shown to integrate multisensory information in a statistically-optimal fashion by averaging the individual unisensory estimates according to their relative reliabilities. This form of integration is optimal in that it yields the most reliable (i.e. least variable) multisensory percept. The present study investigates the neural mechanisms underlying integration of visual and tactile shape information at the macroscopic scale of the regional BOLD response. Observers discriminated the shapes of ellipses that were presented bimodally (visual-tactile) or visually alone. A 2×5 factorial design manipulated (i) the presence vs. absence of tactile shape information and (ii) the reliability of the visual shape information (five levels). We then investigated whether regional activations underlying tactile shape discrimination depended on the reliability of visual shape. Indeed, in primary somatosensory cortices (bilateral BA2) and the superior parietal lobe the responses to tactile shape input were increased when the reliability of visual shape information was reduced. Conversely, tactile inputs suppressed visual activations in the right posterior fusiform, when the visual signal was blurred and unreliable. Somatosensory and visual cortices may sustain integration of visual and tactile shape information either via direct connections from visual areas or top-down effects from higher order parietal areas. no notspecified http://www.kyb.tuebingen.mpg.de/ published -1063 15017 18824 15017 18821 15017 1542015017 18826 6402 3 J Schultz 2010-03-00 6 20 R269 R271 Current Biology Recent advances in neuroimaging allow mental states to be inferred from non-invasive data. In a new study, memories of complex events were successfully decoded solely from imaged activation in a memory-related brain structure. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 15017 15422 5679 3 J Schultz T Lennert 2009-05-00 4 45 1314 1328 NeuroImage A lot is known about the neural basis of directing attention based on explicit cues. In real life however, attention shifts are rarely directed by explicit cues but rather generated implicitly, for example on the basis of previous experience with a given situation. Here, we aimed at studying attention shifts dependent on recent trial history. While explicitly cued attention shifts involve activity in cortex of the intraparietal sulcus, whether this region is also involved in shifting attention according to recent history is still unknown. We asked observers to detect targets in a stream of visual stimuli with three feature dimensions: Color, shape and motion. Critically, target occurrence probability was always higher in one stimulus dimension than in the others, and probabilities switched between dimensions over blocks of trials. After each probability switch, target detection times decreased exponentially for high-probability targets and increased for low-probability targets, compatible with gradual shifts in attention dependent on trial history since the switch. BOLD signal in left prefrontal and intraparietal sulcus regions was higher in the early phase after the switch, while anterior cingulate, cuneus, precuneus, temporal and more anterior frontal regions showed more activation later after the switch. These findings are compatible with the engagement of regions involved in the establishment and maintenance of attentional sets. BOLD signal in left intraparietal sulcus correlated with the size of the performance changes consecutive to the detected targets, suggesting that it reflects the size of attention shifts induced by updating target probabilities over recent trial history. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/SchultzLennert_Stream_accepted_withFigs_formatted_5679[0].pdf published 14 15017 15422 5678 3 J Schultz KS Pilz 2009-04-00 3 194 465 475 Experimental Brain Research The ability to perceive facial motion is important to successfully interact in social environments. Previously, imaging studies have investigated neural correlates of facial motion primarily using abstract motion stimuli. Here, we studied how the brain processes natural non-rigid facial motion in direct comparison to static stimuli and matched phase-scrambled controls. As predicted from previous studies, dynamic faces elicit higher responses than static faces in lateral temporal areas corresponding to hMT+/V5 and STS. Interestingly, analyses of individually-defined, static-face-sensitive regions in bilateral fusiform gyrus and left inferior occipital gyrus also respond more to dynamic than static faces. These results suggest integration of form and motion information during the processing of dynamic faces even in ventral temporal and inferior lateral occipital areas. In addition, our results show that dynamic stimuli are a robust tool to localize areas related to the processing of static and dynamic face info rmation. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/SchultzPilz_facelocaliser_acceptedWithFigs_5678[0].pdf published 10 15017 15422 4686 3 J Schultz L Chuang QC Vuong 2008-06-00 6 18 1302 1313 Cerebral Cortex Shape perception is important for object recognition. However, behavioural studies have shown that rigid motion also contributes directly to the recognition process, in addition to providing visual cues to shape. Using psychophysics and functional brain imaging, we investigated the neural mechanisms involved in shape and motion processing for dynamic object recognition. Observers discriminated between pairs of rotating novel objects in which the three-dimensional shape difference between the pair was systematically varied in metric steps. In addition, the objects rotated in either the same or different direction to determine the effect of task-irrelevant motion on behaviour and neural activity. We found that observers’ shape discrimination performance increased systematically with shape differences, as did the haemodynamic responses of occipito-temporal, parietal and frontal regions. Furthermore, responses in occipital regions were only correlated with observers’ perceived shape differences. We also found d ifferent effects of object motion on shape discrimination across observers which were reflected in responses of the superior temporal sulcus. These results suggest a network of regions that are involved in the discrimination of metric shape differences for dynamic object recognition. no notspecified http://www.kyb.tuebingen.mpg.de/ published 11 15017 15422 3063 3 J Schultz K Friston D Wolpert C Frith 2005-02-00 4 45 625 635 Neuron An essential, evolutionarily stable feature of brain function is the detection of animate entities, and one of the main cues to identify them is their movement. We developed a model of a simple interaction between two objects, in which an increase of the correlation between their movements varied the amount of interactivity and animacy observers attributed to them. Functional magnetic resonance imaging revealed that activation in the posterior superior temporal sulcus and gyrus (pSTS / pSTG) increased in relation to the degree of correlated motion between the two objects. This activation increase was not different when subjects performed an explicit or implicit task while observing these interacting objects. These data suggest that the pSTS and pSTG play a role in the automatic identification of animate entities, by responding directly to an objective movement characteristic inducing the percept of animacy, such as the amount of interactivity between two moving objects. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf3063.pdf published 10 3062 3 J Schultz H Imamizu M Kawato C Frith 2004-12-00 10 16 1695 1705 Journal of Cognitive Neuroscience Previous functional imaging experiments in humans showed activation increases in the posterior superior temporal gyrus and sulcus during observation of geometrical shapes whose movements appear intentional or goal-directed. We modeled a chase scenario between two objects, in which the chasing object used different strategies to reach the target object: the chaser either followed the target’s path or appeared to predict its end position. Activation in the superior temporal gyrus of human observers was greater when the chaser adopted a predict rather than a follow strategy. Attending to the chaser’s strategy induced slightly greater activation in the left superior temporal gyrus than attending to the outcome of the chase. These data implicate the superior temporal gyrus in the identification of objects displaying complex goal-directed motion. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/pdf3062.pdf published 10 3064 3 J Schultz N Sebanz C Frith 2004-10-00 5 27 674 675 Behavioural and Brain Sciences We suggest that certain experiences reported by patients with schizophrenia show that priority, consistency, and exclusivity are not sufficient for the experience of willing an action. Furthermore, we argue that even if priority, consistency, and exclusivity cause the experience of being the author of an action, this does not mean that conscious will is an illusion. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/PrecisWegnerCommentBBS2004_3064[0].pdf published 1 3061 3 J O'Doherty P Dayan J Schultz R Deichmann K Friston RJ Dolan 2004-04-00 5669 304 452 454 Science Instrumental conditioning studies how animals and humans choose actions appropriate to the affective structure of an environment. According to recent reinforcement learning models, two distinct components are involved: a "critic," which learns to predict future reward, and an "actor," which maintains information about the rewarding outcomes of actions to enable better ones to be chosen more frequently. We scanned human participants with functional magnetic resonance imaging while they engaged in instrumental conditioning. Our results suggest partly dissociable contributions of the ventral and dorsal striatum, with the former corresponding to the critic and the latter corresponding to the actor. no notspecified http://www.kyb.tuebingen.mpg.de/ published 2 DobsBBVCS2013 7 K Dobs I Bülthoff M Breidt Q Vuong C Curio J Schultz Bremen. Germany2013-08-00 36th European Conference on Visual Perception (ECVP 2013) no notspecified http://www.kyb.tuebingen.mpg.de/ accepted 0 15017 15422 EsinsSKWB2012_2 7 J Esins J Schultz BR Kim C Wallraven I Bülthoff Schramberg, Germany2012-11-00 38 13th Conference of the Junior Neuroscientists of Tübingen (NeNA 2012) Congenital prosopagnosia, an innate impairment in recognizing faces, as well as the otherrace-effect, the disadvantage in recognizing faces of foreign races, both influence face recognition abilities. Here we compared both phenomena by testing three groups: German congenital prosopagnosics (cPs), unimpaired German and unimpaired South Korean participants (n=23 per group), on three tests with Caucasian faces. First we ran the Cambridge Face Memory Test (Duchaine & Nakayama, 2006 Neuropsychologia 44 576-585). Participants had to recognize Caucasian target faces in a 3AFC task. German controls performed better than Koreans (p=0.009) who performed better than prosopagnosics (p=0.0001). Variation of the individual performances was larger for cPs than for Koreans (p = 0.028). In the second experiment, participants rated the similarity of Caucasian faces (in-house 3D face-database) which differed parametrically in features or second order relations (configuration). We found differences between sensitivities to change type (featural or configural, p=0) and between groups (p=0.005) and an interaction between both factors (p = 0.019). During the third experiment, participants had to learn exemplars of artificial objects (greebles), natural objects (shells), and faces and recognize them among distractors. The results showed an interaction (p = 0.005) between stimulus type and participant group: cPs where better for non-face stimuli and worse for face stimuli than the other groups. Our results suggest that congenital prosopagnosia and the other-race-effect affect face perception in different ways. The broad range in performance for the cPs directs the focus of our future research towards looking for different forms of congenital prosopagnosia. no notspecified http://www.kyb.tuebingen.mpg.de/ published -38 15017 15422 EsinsBKS2012 7 J Esins I Bülthoff I Kennerknecht J Schultz Alghero, Italy2012-09-00 113 35th European Conference on Visual Perception Congenital prosopagnosia, the innate impairment in recognizing faces exhibits diverse deficits. Due to this heterogeneity the possible existence of subgroups of the impairment was suggested (eg Kress and Daum, 2003 Behavioural Neurology14109-21). We examined 23 congenital prosopagnosics (cPAs) identified via a screening questionnaire (as used in Stollhoff, Jost, Elze, and Kennerknecht, 2011 PLoS ONE6e15702) and 23 age-, gender and educationally matched controls with a battery consisting of nine different tests. These included well known tests like the Cambridge Face Memory Test (CFMT, Duchaine and Nakayama, 2006 Neuropsychologia44576-85), a Famous Face Test (FFT), and new, own tests about object and face recognition. As expected, cPAs had lower CFMT and FFT scores than the controls. Analyses of the performance patterns across the nine tests suggest the existence of subgroups within both cPAs and controls. These groups could not be revealed only based on the CFMT and FFT scores, indicating the necessity of tests addressing different, specific aspects of object and face perception for the identification of subgroups. Current work focuses on characterizing the subgroups and identifying the most useful tests. no notspecified http://www.kyb.tuebingen.mpg.de/ published -113 15017 15422 KaulardSWBd2012 7 K Kaulard J Schultz C Wallraven HH Bülthoff S de la Rosa Alghero, Italy2012-09-00 103 35th European Conference on Visual Perception The face inversion effect has often been demonstrated in face identification tasks. Less is known about whether processes underlying face expression recognition are also sensitive to face inversion. Face expression recognition is usually investigated using pictures of six emotional expressions. In everyday life, humans are however exposed to a much larger set of facial expressions, which are dynamic. Here, we examine the effect of face inversion on expression recognition for a variety of facial expressions displayed statically and dynamically. We measured participants'recognition accuracy for 12 expressions using a 13 alternative-forced-choice task. We varied the dynamics (videos versus pictures) and the orientation (upright versus inverted) of the presentation of the expressions in a completely crossed design. Accuracy was significantly higher when expressions were presented as videos (62%) than as pictures (47%). Similarly, recognition accuracy was significantly higher for upright (84%) compared to inverted (64%) expressions. Moreover, the effect of orientation changed significantly with expression type. No other effects were significant. This is the first study to report that face inversion affects the recognition of natural facial expressions. Because face inversion effects are interpreted as a sign of configural processing, our results suggest configural processing for a majority of facial expressions. no notspecified http://www.kyb.tuebingen.mpg.de/ published -103 15017 15422 DobsBCS2012 7 K Dobs I Bülthoff C Curio J Schultz Naples, FL, USA2012-08-00 35 12th Annual Meeting of the Vision Sciences Society (VSS 2012) Previous research has shown that facial motion can convey information about identity in addition to facial form (e.g. Hill & Johnston, 2001). The present study aims at finding whether identity judgments vary depending on the kinds of facial movements and the task performed. To this end, we used a recent facial motion capture and animation system (Curio et al., 2006). We recorded different actors performing classic emotional facial movements (e.g. happy, sad) and non-emotional facial movements occurring in social interactions (e.g. greetings, farewell). Only non-rigid components of these facial movements were used to animate one single avatar head. In a between-subject design, four groups of participants performed identity judgments based on emotional or social facial movements in a same-different (SD) or a delayed matching-to-sample task (XAB). In the SD task, participants watched two distinct facial movements (e.g. happy and sad) and had to choose whether the same or different actors performed these facial movements. In the XAB task, participants saw one target facial movement X (e.g. happy) performed by one actor followed by two facial movements of another kind (e.g. sad) performed by two actors. Participants chose which of the latter facial movements was performed by the same actor as the one performing X. Prior to the experiment, participants were familiarized with the actors by watching them perform facial movements not subsequently tested. Participants were able to judge actor identities correctly in all conditions, except for the SD task performed on the emotional stimuli. Sensitivity to identity as measured by d-prime was higher in the XAB than in the SD task. Furthermore, performance was higher for social than for emotional stimuli. Our findings reveal an effect of task on identity judgments based on facial motion, and suggest that such judgments are easier when facial movements are less stereotypical. no notspecified http://www.kyb.tuebingen.mpg.de/ published -35 15017 15422 EsinsSKWB2012 7 J Esins J Schultz BR Kim C Wallraven I Bülthoff Incheon, South Korea2012-07-00 57 8th Asia-Pacific Conference on Vision (APCV 2012) Congenital prosopagnosia, an innate impairment in recognizing faces, as well as the other-race-effect, the disadvantage in recognizing faces of foreign races, both influence face recognition abilities. Here we compared both phenomena by testing three groups: German congenital prosopagnosics (cPs), unimpaired German and unimpaired South Korean participants (n=23 per group), on three tests with Caucasian faces. First we ran the Cambridge Face Memory Test (Duchaine & Nakayama, 2006 Neuropsychologia 44 576-585). Participants had to recognize Caucasian target faces in a 3AFC task. German controls performed better than Koreans (p=0.009) who performed better than prosopagnosics (p=0.0001). Variation of the individual performances was larger for cPs than for Koreans (p = 0.028). In the second experiment, participants rated the similarity of Caucasian faces (in-house 3D face-database) which differed parametrically in features or second order relations (configuration). We found differences between sensitivities to change type (featural or configural, p=0) and between groups (p=0.005) and an interaction between both factors (p = 0.019). During the third experiment, participants had to learn exemplars of artificial objects (greebles), natural objects (shells), and faces and recognize them among distractors. The results showed an interaction (p = 0.005) between stimulus type and participant group: cPs where better for non-face stimuli and worse for face stimuli than the other groups. Our results suggest that congenital prosopagnosia and the other-race-effect affect face perception in different ways. The broad range in performance for the cPs directs the focus of our future research towards looking for different forms of congenital prosopagnosia. no notspecified http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/2012/APCV-2012-Poster-Esins.pdf published -57 15017 15422 PapeWSBM2011 7 A-A Pape T Wolbers J Schultz HH Bülthoff T Meilinger Washington, DC, USA2011-11-00 41st Annual Meeting of the Society for Neuroscience (Neuroscience 2011) Grid cells in entorhinal cortex of freely moving rodents were proposed to provide a universal metric of space. They tile the environment into a six-fold symmetric pattern with a particular orientation relative to the environment. The six-fold rotational symmetry of grid patterns can be used to predict a macroscopic signal to functional magnetic resonance imaging (fMRI) in humans [Doeller et al, 2010, Nature]. During hippocampal remapping, grid pattern orientations in rats also change. The purpose of the present study is to examine whether orientation changes (i.e., remapping) can also be found in humans. Participants learned object locations within a virtual room (see Figure 1 left side) and retrieved locations from different start locations during two scanning sessions. They then navigated into an adjacent room and repeated the procedure. We extracted grid orientations from odd trials, and predicted the BOLD response in even trials as a function of the deviation between running direction and the estimated grid orientation for each session. This prediction was significant for the right entorhinal cortex, replicating earlier findings. In 80% of the cases grid cell orientations significantly differed between sessions both within a room and between rooms (see Figure 1 right side). Switching off the virtual environment between sessions for about one minute was seemingly sufficient for that. For male, but not for female participants, grid cell orientation was clustered around the random view of the room experienced at session start. Data suggests that human grid cell orientations can be rather flexible which might be due to the virtuality of the experience. Grid cell orientation might at least for male participants be related to the initial view of an environment. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 15017 15422 PapeWSBM2011_2 7 A-A Pape T Wolbers J Schultz HH Bülthoff T Meilinger Heiligkreuztal, Germany2011-10-00 38 12th Conference of Junior Neuroscientists of Tübingen (NeNA 2011) Grid cells in entorhinal cortex of freely moving rodents were proposed to provide a universal metric of space. They tile the environment into a six-fold symmetric pattern with a particular orientation relative to the environment. The six-fold rotational symmetry of grid patterns can be used to predict a macroscopic signal to functional magnetic resonance imaging (fMRI) in humans [Doeller et al, 2010, Nature]. During hippocampal remapping, grid pattern orientations in rats also change. The purpose of the present study is to examine whether orientation changes (i.e. remapping) can also be found in humans. Participants learned object locations within a virtual room and retrieved locations from different start locations during two scanning sessions. They then navigated into an adjacent room and repeated the procedure. We extracted grid orientations from odd trials, and predicted the BOLD response in even trials as a function of the deviation between running direction and the estimated grid orientation for each session. This prediction was significant for the right entorhinal cortex, replicating earlier findings. In 80% of the cases grid cell orientations significantly differed between sessions both within a room and between rooms. Switching off the virtual environment between sessions for about one minute was seemingly sufficient for that. For male, but not for female participants, grid cell orientation was clustered around the random view of the room experienced at session start. Data suggests that human grid cell orientations can be rather flexible which might be due to the virtuality of the experience. Grid cell orientation might at least for male participants be related to the initial view of an environment. no notspecified http://www.kyb.tuebingen.mpg.de/ published -38 15017 15422 SchultzB2011 7 J Schultz HH Bülthoff Naples, FL, USA2011-09-00 682 11th Annual Meeting of the Vision Sciences Society (VSS 2011) Identifying living moving things in the environment is a priority for animals, as these could be prey, enemies or mates. When the shape of the moving object is hard to see (fog, twilight, great distance, small animal), motion becomes an important cue to detect it. The neural correlates of the detection of an isolated living entity on the basis of its motion are largely unknown. To study this phenomenon, we developed a single-dot stimulus, thus eliminating all possible sources of information about form, spatial arrangement, shape or structure of the object. The dot moved such that it appeared self-propelled, or moved by an external force, or something intermediary according to a small set of parameters. Self-propelled stimuli were perceived as more animate (= more likely to be alive) than the externally-moved stimuli, with a gradual transition occurring in the intermediary morphs following a classic psychometric function (cumulative gaussian). In an fMRI experiment, 20 subjects had to categorize these stimuli into alive and non-alive. A region of the left medial posterior parietal cortex (mPPC) showed BOLD signal correlating with the probability of animacy judgments about the moving dot. While activation in parts of the early visual cortex showed the same response, the mPPC was the only region in which changes in percept had a stronger effect on activation than physical changes in the stimuli. In addition, only the mPPC showed BOLD signal increases when a stimulus was judged to be animate, irrespective of its physical characteristics. This study shows that parts of the early visual cortex but particularly the medial posterior parietal cortex (mPPC) are involved in judging the animacy of an isolated translating visual stimulus, without information about its form. no notspecified http://www.kyb.tuebingen.mpg.de/ published -682 15017 15422 DobsKBSC2011 7 K Dobs M Kleiner I Bülthoff J Schultz C Curio Toulouse, France2011-09-00 115 34th European Conference on Visual Perception 3D facial animation systems allow the creation of well-controlled stimuli to study face processing. Despite this high level of control, such stimuli often lack naturalness due to artificial facial dynamics (eg linear morphing). The present study investigates the extent to which human visual perception can be fooled by artificial facial motion. We used a system that decomposes facial motion capture data into time courses of basic action shapes (Curio et al, 2006 APGV 1 77–84). Motion capture data from four short facial expressions were input to the system. The resulting time courses and five approximations were retargeted onto a 3D avatar head using basic action shapes created manually in Poser. Sensitivity to the subtle modifications was measured in a matching task using video sequences of the actor performing the corresponding expressions as target. Participants were able to identify the unmodified retargeted facial motion above chance level under all conditions. Furthermore, matching performance for the different approximations varied with expression. Our findings highlight the sensitivity of human perception for subtle facial dynamics. Moreover, the action shape-based system will allow us to further investigate the perception of idiosyncratic facial motion using well-controlled facial animation stimuli. no notspecified http://www.kyb.tuebingen.mpg.de/ published -115 15017 15422 EsinsBS2011 7 J Esins I Bülthoff J Schultz Naples, FL, USA2011-09-00 673 11th Annual Meeting of the Vision Sciences Society (VSS 2011) An important aspect of face recognition involves the role of featural and configurational information for face perception (e.g. Tanaka and Farah, 1993; Yovel and Duchaine, 2006; Rotshtein et al, 2007). In our study, we investigated the influence of featural and configural information on perceived similarity between faces. Eight pairs of male faces were chosen from our digital face database (http://faces.kyb.tuebingen.mpg.de). The texture and the face shape for both faces in a pair were equalized to create 2 basis faces that differed only in their inner facial features and their configuration, but not in face shape or texture. A computer algorithm allowed to parametrically morph the features, the configuration, or both between the two basis faces of a pair. In our case the morphing was done in 25% steps. 24 participants rated the similarity between pairs of the created faces using a 7-point Likert scale. The faces to compare came from the same basis face pair and could differ either in features or in configuration by 0%, 25%, 50%, 75% or 100%. The results revealed that for the same amount of morphing, faces differing by their features are perceived as less similar than faces differing by their configurations. These findings replicate previous results obtained with less natural or less controlled conditions. Furthermore, we found that linear increases of the difference between both faces in configural or featural information resulted in a nonlinear increase of perceived dissimilarity. An important aspect for the relevance of our results is how natural the face stimuli look like. We asked 24 participants to rate the naturalness of all stimuli including the original faces and the created faces. Despite numerous manipulations, the vast majority of our created face stimuli were rated as natural as the original faces. no notspecified http://www.kyb.tuebingen.mpg.de/fileadmin/user_upload/files/publications/2011/VSS-2011-Esins.pdf published -673 15017 15422 KaulardFBS2011 7 K Kaulard AL Fernandez Cruz HH Bülthoff J Schultz Naples, FL, USA2011-09-00 605 11th Annual Meeting of the Vision Sciences Society (VSS 2011) Facial expressions convey both emotional and conversational signals. Research focuses mostly on EMOTIONAL expressions and consistenly finds that these can be reliably distinguished along at least two dimensions: valence and arousal. CONVERSATIONAL expressions, i.e. those conveying mainly communicative meaning, are thought to be less emotional laden. Interestingly, we found evidence pointing towards the same first two underlying dimensions for CONVERSATIONAL expressions when presented dynamically. The question now arises: "Is the emergence of the valence and arousal dimensions for conversational facial expressions based on the emotional content of these expressions?" To answer this, we used questions addressing the emotional (Fontaine et al, 2007) and the conversational content separately. If the distinction of conversational expressions is based on the small amount of emotional information they might contain, we expect emotional content questions to allow a separation of those expressions. Ten native German participants answered a set of 27 questions for 6 emotional and 6 conversational expressions, both presented statically and dynamically, using a rating scale. A dissimilarity matrix was computed for the expressions. To uncover the meaning of the first two underlying dimensions allowing expression differentiation, multidimensional scaling (MDS) was used. Our results show that static and dynamic emotional expressions can only be distinguished by means of emotional content questions. For these emotional expressions, the valence and arousal dimensions emerged in the MDS. In contrast, conversational expressions can be distinguished using conversational content questions but not using emotional content questions. Unlike emotional expressions, dynamic information improved distinction of conversational expressions substantially. We found evidence for valence and arousal to be the underlying dimensions for conversational expressions. Our results suggest that the distinction of conversational expressions along the first two dimensions is based on conversational rather than emotional content. Moreover, different types of facial expressions benefit to different degrees from dynamic information. no notspecified http://www.kyb.tuebingen.mpg.de/ published -605 15017 15422 KaularddSFBW2011 7 K Kaulard S de la Rosa J Schultz AL Fernandez Cruz HH Bülthoff C Wallraven Toulouse, France2011-09-00 115 34th European Conference on Visual Perception Similarity ratings are used to investigate the cognitive representation of facial expressions. The perceptual and cognitive properties (eg physical aspects, motor expressions, action tendencies) driving the similarity judgments of facial expressions are largely unknown. We examined potentially important properties with 27 questions addressing the emotional and conversational content of expressions (semantic differential). The ratings of these semantic differentials were used as predictors for facial expression similarity ratings. The semantic differential and similarity-rating task were performed on the same set of facial expression videos: 6 types of emotional (eg happy) and 6 types of conversational (eg don’t understand) expressions. Different sets of participants performed the two tasks. Multiple regression was used to predict the similarity data from the semantic differential questions. The best model for emotional expressions consisted of two emotional questions explaining 75% of the variation in similarity ratings. The same model explained significantly less variation for conversational expressions (38%). The best model for those expressions consisted of a single conversational question explaining 44% of the variation. This study shows which properties of facial expressions might affect their perceived similarity. Moreover, our results suggest that different perceptual and cognitive properties might underlie similarity judgments about emotional and conversational expressions. no notspecified http://www.kyb.tuebingen.mpg.de/ published -115 15017 15422 SchultzBP2011 7 J Schultz M Brockhaus K Pilz Toulouse, France2011-09-00 116 34th European Conference on Visual Perception Visual perception of moving faces activates parts of the human superior temporal sulcus (STS) whereas static facial information is mainly processed in areas of ventral temporal and lateral occipital cortex. However, recent findings show that the latter regions also respond more to moving faces than to static faces. Here, we investigated the origin of this activation increase, considering the following causes: (i) facial motion per se, (ii) increased static information due to the higher number of frames constituting the movie stimuli, and/or (iii) increased attention towards moving faces. We presented non-rigidly moving faces to subjects in an fMRI scanner. We manipulated static face information and motion fluidity by presenting ordered and scrambled sequences of frames at the original or reduced temporal resolutions. Subjects performed a detection task unrelated to the face stimuli in order to equate attentional influences. Results confirm the increased response due to facial motion in the face-sensitive temporal regions. Activation generally increased with the number of frames but decreased when frames were scrambled. These results indicate that the activation increase induced by moving faces is due to smooth, natural motion and not only to increased static information or attentional modulation. no notspecified http://www.kyb.tuebingen.mpg.de/ published -116 15017 15422 DavidSVE2011_2 7 N David J Schultz K Vogeley A Engel Cambridge, UK2011-04-00 The Social Brain Workshop 2011 no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 15017 15422 DavidSVE2011 7 N David J Schultz K Vogeley A Engel San Francisco, CA, USA2011-03-00 64 18th Annual Meeting of the Cognitive Neuroscience Society (CNS 2011) A focus on social deficits in autism spectrum disorders (ASD) has, for a long time, obscured the existence of lower-level perceptual abnormalities, although the earliest descriptions of autism included abnormalities in oculomotor behavior and visual attention. More recently, however, abnormalities in perception and attention have increasingly been discussed as influential factors in ASD-specific psychopathology. To this end, the perception of coherent motion in random-dot kinematograms, biological motion in point-light walkers and agency in animated shapes have been investigated in ASD but their relationship remains a matter of debate. It also is unclear whether ASD-related deficits result from difficulties in global motion perception or in processing motion that contains socially relevant signals (e.g. a body and actions). We tested 18 individuals with highfunctioning autism and 16 age-, gender- and IQ-matched control participants, who performed three tasks on a continuum of motion cues and social complexity: (1) low-level translational motion that moved up or down, (2) complex motion of a single dot that moved in an animate or inanimate way, (3) complex motion of two dots that interacted or not. None of these tasks contained objects with human shape and only the first task contained global motion. Participants with autism were selectively impaired in detecting social interaction between two animated shapes (task 3), while low-level motion processing (task 1) and the detection of isolated agents (task 2) were preserved. These findings suggest a distinct social impairment in ASD in understanding interacting agents. no notspecified http://www.kyb.tuebingen.mpg.de/ published -64 15017 15422 7068 7 GR Sigala Alanis J Schultz NK Logothetis G Rainer San Diego, CA, USA2010-11-00 40th Annual Meeting of the Society for Neuroscience (Neuroscience 2010) While face categorization is a fundamental cognitive ability of human and non-human primates, its neural basis remain poorly understood. Using a new morphing technique, we created realistic three-dimensional morphed faces that linearly span the continuum between humans and monkeys (“species” continuum). Extensive categorization and discrimination experiments in human observers show that humans perceive the “species” continuum categorically. Moreover, the position of the categorical boundary is shifted from the center towards the human end of the continuum, suggesting a higher sensitivity to changes near the own-species prototype. We presented a subset of these faces to human subjects in a block-design fMRI experiment to record BOLD signals from the temporal lobe while participants performed an unrelated task at fixation. We applied a multivariate approach based on (Pearson) correlations to compute the difference between activity patterns elicited by faces along the continuum. Using this method, we looked for a categorical representation in face selective areas previously defined using an independent, standard "Face-localizer" experiment. Consistent with the psychophysical results, we found a categorical response with a bias towards the human end of the stimulus continuum in the activation patterns of the left human STS. In addition, activation in human ventral temporal cortex was most sensitive to deviations from the human prototype. To look for similar effects in monkeys, we applied an equivalent multivariate approach to analyze extracellular signals from a population of neurons recorded from the STS of two macaque monkeys while they fixated at the same type of faces. Additionally, the position of the perceptual category boundary was determined with a preferential-looking-time experiment. In both behavioral and neuronal monkey data, we found a categorical representation of the continuum, but in this case, with a bias towards the monkey end of the continuum. Our results demonstrate the neural basis of categorical representation of a facial attribute in the human and non-human primate brain. Together, our findings suggest that experience can lead to significant shifts in category boundary for face stimuli. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 15017 1542115017 15422 SchultzBP2011_2 7 J Schultz M Brockhaus K Pilz San Diego, CA, USA2010-11-00 40th Annual Meeting of the Society for Neuroscience (Neuroscience 2010) Visual perception of moving faces activates parts of the human superior temporal sulcus (STS) whereas static facial information is mainly processed in areas of ventral temporal and lateral occipital cortex. However, recent findings show that the latter regions also respond more to moving faces than to static faces (Schultz and Pilz, 2009). This study investigated the origin of this activation increase. We considered the following causes: (1) facial motion per se (2) increased static information due to the higher number of frames constituting the movie stimuli, and/or (3) increased attention towards moving faces, which would increase the response in face-sensitive areas through top-down modulation. We presented non-rigidly moving faces to participants lying in an fMRI scanner. We manipulated static face information and fluidity of the motion in the stimuli by presenting ordered and scrambled sequences of frames at temporal resolutions of 25 frames per second (fps) which were perceived as fluid motion when ordered, but as very non-fluid when scrambled, 12.5 fps (still perceived as fluid when ordered) and 5 fps (significantly less fluid when ordered, with smallest effect of scrambling). To control the influence of attention, subjects were asked to perform a target detection task that was unrelated to the face stimuli (one-back matching task on a stream of letters presented at fixation). Results confirm the increased activation induced by facial motion in the face-sensitive fusiform and superior temporal regions. A purely attention-based effect can be ruled out given that the task performance was far from ceiling performance and equal across conditions. While activation generally increased with the number of frames, a significant reduction of activation was observed due to frame-scrambling the stimuli. These results indicate that the activation increase induced by moving faces is due to the motion of the stimulus (i.e. temporal order of the frames) and not only to increased static information or attentional modulation. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 15017 15422 Schultz2010 7 J Schultz Lausanne, Switzerland2010-08-00 19 33rd European Conference on Visual Perception We previously developped stimuli allowing parametric control over the percept of animacy evoked by the movements of a single object, without contribution from spatial arrangement, shape or structure of the object (Schultz and Dopjans, 2008 Perception 35 ECVP Supplement, 154). As observers tend to follow the stimulus with their eyes while performing animacy judgments, we quantified these eye movements in the present study (Experiment 1). In Experiment 2, we tested the importance of eye movements and attention for task performance by forcing subjects to fixate while judging animacy. In Experiment 3, attentional resources were further reduced by asking subjects to perform a secondary task at fixation while judging animacy. Experiment 1 showed that the distance between eye fixations and the stimulus increased with changes in animacy, compatible with a greater difficulty in following animate-looking stimuli. Combined results across experiments show that the strength of the changes in percept tends to be reduced with fixed gaze and is significantly decreased in the dual-task setting. In the latter, the greatest disruption in stimulus processing appears to result from detecting and reporting the fixation targets rather than just splitting attentional resources. These results suggest that at least partially sustained attention is required for animacy judgments about our single moving dot stimulus. no notspecified http://www.kyb.tuebingen.mpg.de/ published -19 15017 15422 6751 7 F Schillinger S de la Rosa J Schultz K Uludag Lausanne, Switzerland2010-08-00 54 33rd European Conference on Visual Perception Mirror Neurons (MN) have been suggested to be the supporting neural mechanism for action recognition and understanding. However, there is a current debate about the localization of MN in humans. Functional magnetic resonance imaging (fMRI) studies using repetition suppression (RS) paradigms for the identification of MN provide mixed results. Studies supporting the existence of MN restricted their analysis to a-priori candidate regions, whereas studies that failed to find evidence used non-object-directed actions. In the present fMRI study, we tackled these limitations by using object-directed actions in a RS paradigm and performing a wholebrain analysis. Subjects observed and executed simple grasping movements differing only in their goal-directness (grasping a button vs. grasping beside it). MN areas should be (1) more activated by goal-directed actions and (2) exhibit RS between execution and observation of the same action. The analysis revealed three significant cortical clusters in the right anterior intraparietal sulcus (aIPS), right primary somatosensory cortex and left premotor cortex that show these characteristics. While the aIPS has been reported before as a possible region for MN, the other two clusters haven’t been associated with MN directly yet using RS paradigms. We discuss the potential contribution of these regions to object-directed actions. no notspecified http://www.kyb.tuebingen.mpg.de/ published -54 15017 1542215017 18821 6818 7 R Sigala J Schultz NK Logothetis G Rainer Santorini, Greece2010-06-00 93 AREADNE 2010: Research in Encoding And Decoding of Neural Ensembles Categorization of faces is fundamental for social interactions of primates. To understand its neural basis, we investigate how human and monkey face categories are represented in both the human and non-human primate brain. As stimuli, we use realistic three-dimensional morphed faces that linearly span the continuum between humans and monkeys (Fig. 1A). Extensive behavioral tests in both species revealed categorical perception with a shift of the categorical boundary towards the own species (Fig. 1B). This suggests that both species perceive the same stimulus continuum in a fundamentally different way. During a fixation task, we recorded from the temporal lobe extracellular signals in monkeys and BOLD signals in humans. To analyze the data, we used a multivariate pattern classifier approach based on Support Vector Machines and correlations. Consistent with the psychophysical results, we found an "own-species" bias in the categorical representation of human and monkey faces at the level of single neurons as well as in the population response in the inferior temporal lobe of the monkey. (Fig. 1C). Symmetrically, we found a categorical response with an ownspecies bias in the activation patterns of the left human STS. In addition, human ventral temporal cortex showed a higher sensitivity for human faces. Our results are the first to demonstrate the neural basis of categorical representation of a facial attribute in the primate brain. In addition, our data show that both psychophysical and neuronal data can show categorical boundary shifts indicative of the behavioral relevance of prototypical categories. no notspecified http://www.kyb.tuebingen.mpg.de/ published -93 15017 1542115017 15422 SchultzB2010 7 J Schultz HH Bülthoff Barcelona, Spain2010-06-00 62 16th Annual Meeting of the Organisation for Human Brain Mapping (HBM 2010) no notspecified http://www.kyb.tuebingen.mpg.de/ published -62 15017 15422 6291 7 J Schultz HH Bülthoff Chicago, IL, USA2009-10-00 39th Annual Meeting of the Society for Neuroscience (Neuroscience 2009) Animals (including humans) have to identify living moving things in the environment: these could be prey, enemies or mates and interactions with them should be actively controlled. Living things could be detected visually through their shape or their motion, or both. When shape is hard to see (fog, twilight, great distance, small animal), motion becomes an important cue. Biological motion has been studied widely using point-light displays, but these displays appear to contain some sort of shape or form information that influences recognition. To study the neural correlates of the detection of living entities from motion alone, we developed a stimulus consisting of a single moving dot, thus eliminating all possible sources of information about form, spatial arrangement, shape or structure of the object. Our single dot moved such that it appeared either self-propelled (modelled on the movements of a fly) or moved by an external force (modelled on a leaf drifting in the wind). Both types of movement were built using the same equation but differed in speed and acceleration profiles according to a small set of parameters. Low-level stimulus characteristics of the stimuli (range of positions on the screen, average speed, overall aspect of the trajectory) were kept as constant as possible. The parameters could be varied in a continuous fashion to create morphs between the self-propelled and externally-moved extremes. Consistent with expectations, behavioral experiments showed that self-propelled stimuli were perceived as more animate (= more likely to be alive) than the externally-moved stimuli, with a gradual transition occurring in the intermediary morphs. The extreme stimuli and four intermediary morphs were presented in an fMRI experiment to participants who had to categorize the stimuli into alive and non-alive. Using separate functional localizers, we located areas hMT+/V5 and the superior temporal sulcus region responding to point-light walkers, and found that neither region showed changes in BOLD response following the changes in percept. However, BOLD response in a region of the left posterior superior parietal cortex scaled with the degree of perceived animacy. This suggests that the STS is not simply a detector of all kinds of animate motion, but might only be implicated when some sort of shape information in the stimuli (as with point-light displays or with interacting dots) is contributing to the percept of animacy. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 15017 15422 SchultzL2009 7 J Schultz T Lennert Regensburg, Germany2009-08-00 137 32nd European Conference on Visual Perception A lot is known about the neural basis of directing attention based on explicit cues. In real life however, attention shifts are rarely directed by explicit cues but rather generated implicitly, for example on the basis of previous experience. Here, we aimed at studying attention shifts dependent on recent trial history. We asked observers to detect targets in a stream of visual stimuli with three feature dimensions: colour, shape and motion. Critically, target occurrence probability was always higher in one stimulus dimension than in the others, and probabilities switched between dimensions over blocks of trials. After each probability switch, target detection times decreased exponentially for high-probability targets and increased for low-probability targets, compatible with gradual shifts in attention dependent on trial history since the switch. BOLD signal in left prefrontal and intraparietal sulcus regions was higher in the early phase after the switch, while anterior cingulate, cuneus, precuneus, temporal and more anterior frontal regions showed more activation later after the switch. These findings are compatible with expectation about engagement of regions involved in the establishment and maintenance of attentional sets. BOLD signal in left intraparietal sulcus correlated with the size of the performance changes consecutive to the detected targets, suggesting that it reflects the size of attention shifts induced by updating target probabilities over recent trial history. no notspecified http://www.kyb.tuebingen.mpg.de/ published -137 15017 15422 5680 7 J Schultz L Dopjans Utrecht, Netherlands2008-08-00 154 31st European Conference on Visual Perception Humans attribute animacy even to very simple objects displaying self-propelled or goal-directed motion. To test attribution of animacy parametrically using classical psychophysical techniques, we created animations consisting of a single dot that appeared either self-propelled (modelled on the movements of a fly) or moved by an external force (modelled on a leaf drifting in the wind). Both animations were built using the same movement equation and differed in speed and acceleration profiles, allowing parametric morphing from one &lsquo;extreme&lsquo; animation to the other. Low-level stimulus properties (range of screen positions covered, speed or acceleration) did not vary systematically during morphing. 26 naive subjects were asked to rate the &lsquo;extreme&lsquo; animations and 4 intermediate morphs for animacy. Ratings from 19 subjects as well as averages over all subjects could be modelled by a cumulative Gaussian, median PSE was in the middle of the morph range and the median JND was 1.7. These stimuli thus allow parametric testing of animacy perception from single objects with movements modelled on real animate entities. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/jschultzECVP2008_v2_[0].pdf published -154 15017 15422 4687 7 J Schultz MX Cohen S Haupt HH Bülthoff C Elger San Diego, CA, USA2007-11-00 37th Annual Meeting of the Society for Neuroscience (Neuroscience 2007) An essential need of brain function is the detection of living entities, and one of the major characteristics for their identification is their motion. Humans are very good at recognizing living entities from their motion, and attribute animacy to even very simple objects displaying self-propelled or goal-directed motion. Our previous results (1) show that increasing correlation between the movements of two simple interacting objects leads to A) an increase in the impression of goal-directed motion and of animacy and B) to increasing BOLD signal in the superior temporal sulcus (STS), suggesting that STS is involved in decoding the information leading to the percept of animacy. This is consistent with previous studies implicating the posterior part of the STS in recognition of biological motion. In the current study, 7 volunteering patients undergoing investigations prior to epilepsy surgery observed the animate motion stimuli used in (1). Simultaneously, we recorded multichannel subdural electrocorticogram data from healthy cortex surrounding the STS and performed trial-by-trial frequency decomposition over time. We found that power in the 30-60Hz frequency band (gamma band) between 1 and 2 seconds after stimulus onset showed a significant parametric response to the amount of goal-directed motion, paralleling our previous BOLD signal findings (1). Furthermore, due to the high temporal resolution of these data, we were able to localize at which time points during the animations the strongest response in the STS occurred. Our results confirm the importance of the STS in processing of visual characteristics of animate entities, and suggest that neuronal activity in this area changes over the duration of the animations. We are currently comparing time-varying attributes of the stimuli to the time-course of gamma-band activity to reveal which events in the stimuli drive STS activity. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 15017 15422 4852 7 J Schultz T Lennert HH Bülthoff Tübingen, Germany2007-07-00 49 10th Tübinger Wahrnehmungskonferenz (TWK 2007) Attention determines which aspects of the incoming sensory information are processed with priority. However, attention is seldom an all-or-none process but rather distributed over multiple kinds of incoming information, and this distribution must be updated according to events in the world. Despite its ubiquity, this dynamic updating has been little studied in psychophysics, and even less is known about its neural correlates. In order to investigate attention updating, we studied serial detection of targets in different dimensions (color, shape or motion) of visual stimuli. Performance changed according to target sequence, and could be explained by this simple behavioral model: Each detected target was followed by a discrete attention shift towards the dimension in which the target occurred, leading to a short-lasting, exponentially decaying performance benefit. Continuously changing performance over time reflected the dynamic updating of attention induced by the sequence of detected targets. BOLD signal predicted by this time-course of attention changes was found exclusively in left parietal cortex, suggesting that neural activity in this area directly reflects how world events influence the distribution of attention. no notspecified http://www.kyb.tuebingen.mpg.de/ published -49 15017 15422 4093 7 J Schultz HH Bülthoff Atlanta, GA, USA2006-10-00 36th Annual Meeting of the Society for Neuroscience (Neuroscience 2006) Temporal patterning of stimuli can affect performance and be critical for perceptual learning. We studied the neural correlates of trial history effects using a task in which detection time was influenced by target history. Using an MRI scanner we measured BOLD signal changes while 12 subjects were presented with streams of stimuli of variable colors, shapes, and motion directions. Participants had to attend to all 3 stimulus dimensions simultaneously to report targets consisting of unpredictable stimulus feature repetitions. Response times for targets in each stimulus dimension decreased exponentially with the number of successive targets and were well explained by a leaky integrator of target history with fast exponential decay (half-life = 1.21 trials). Significant BOLD responses (random-effects analysis over 12 subjects, threshold = p<0.05 corrected for family-wise errors at the cluster level for all reported effects) predicted by theoretical neuronal activity reflecting the leaky integrator output for all stimulus dimensions were found bilaterally in the striatum (putamen, head and body of caudate). In addition, significant BOLD signal increases were observed in response to detected targets of any stimulus dimension in lateral occipital cortex and fusiform gyri bilaterally, with specific responses in regions compatible with area MT for motion targets, with area LO for shape targets and with area V4v for color targets. Our behavioural data show that detected targets induce a benefit in response time for subsequent targets in the same stimulus dimension, that this acceleration effect is short-lived and can be modelled by a leaky integrator of target history. Our fMRI data show 1) BOLD signal increases compatible with neural activity reflecting the leaky integrator signals in striatum, in line with a role of the striatum in guiding motor behaviour in response to sensory cues and 2) BOLD signal increases in extrastriate visual areas in response to detected targets, reflecting immediate sensory consequences of detected targets. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 15017 15422 ConradMS2006 7 V Conrad JS McDonald J Schultz St. Petersburg2006-08-00 101 102 29th European Conference on Visual Perception During continuous viewing of multistable figures, such as the Necker cube, perception alternates between equally valid solutions. So how can perceptual experience be stable given that various alternative interpretations of the same physical stimulus are available? Previous demonstrations with bistable stimuli have revealed that a repetitive intermittent presentation leads to a stabilisation of the percept. Recent research findings suggested that interleaved presentation of several ambiguous stimuli does not disrupt the perceptual stabilisation of each reversible pattern, suggesting that perceptual 'memory stores' coexist independently for each representation. Interference effects were only obtained for structurally similar stimuli. In the present study, we adopted Maier et al's interleaved presentation paradigm to investigate the effects of interfering ambiguous patterns upon transition probability and the stabilisation process. Rather than manipulating structural similarities between interleaved ambiguous stimuli, we sequentially presented ambiguous figures that share equivalent reversal processes such as figure - ground segregation or perspective reversal. The results reveal that perceptual dominance time of the ambiguous test stimulus decreases compared to periods during which a blank interval is presented, indicating an effect of interference from the distractor. Interaction between reversal processes influences the stabilisation of perception that is normally observed during repetitive intermittent presentation with blank intervals. no notspecified http://www.kyb.tuebingen.mpg.de/ published 1 15017 15422 4226 7 QC Vuong J Schultz L Chuang St. Petersburg, Russia2006-08-00 99 29th European Conference on Visual Perception The role of object parts is a key issue in object recognition. Here we investigated whether observers encode qualitative (eg straight versus curved part) or metric information of parts (eg curvature magnitude), and whether the information that is encoded can be affected by motion. To address these issues, we constructed a novel set of objects composed of parts that can vary metrically along different dimensions (eg tapering and bending) to create qualitatively different parts. In a same/different matching task, we presented two objects rigidly rotating in the same or different direction, and had observers judge whether these objects were the same or different. We varied the pair of objects along an &lsquo;identity&lsquo; axis by morphing between two exemplars. A cumulative Gaussian function explained the effect of morph level, suggesting that observers encoded metric information. There was a slight shift of the psychometric function for same versus different motion. Overall, our results suggest that observers are sensitive to metric information, even for objects with salient part structure. We are currently investigating with fMRI how object parts and motion influence neuronal object processing. no notspecified http://www.kyb.tuebingen.mpg.de/ published -99 15017 15422 4828 7 J Schultz F de Vignemont Tübingen, Germany2006-03-00 69 9th Tübingen Perception Conference (TWK 2006) Theory-of-Mind, or mentalising, is defined as a cognitive process used to understand other peoples’ actions based on mental states. Twomain theories of mentalising have been put forward in recent years: Simulation Theory and Theory-Theory. We propose a model of mentalising based on action prediction and semantic representation. The model would be triggered whenever a human observer detects a potential agent (particularly other humans, but also other animals or active entities). On the basis of their actions, it would associate a possible mental state to the observed agent and predict its future behaviour. To do this, first a search engine would look for a potential mental state matching an observed action in a look-up table containing actionmental state associations acquired through experience. Then, a predictor would calculate a possible next action for the observed agent on the basis of the mental state, and a comparator would compare this predicted action to the actual next action of the agent. If the discrepance between predicted and actual behaviour is greater than a threshold, the mental state is rejected and the process repeated until a conclusive match or abandon. The predictor is postulated to be similar to mechanisms thought to underlie motor learning or reinforcement learning, while the look-up table could resemble semantic representations of objects or faces. The model could also be used for active interaction with other agents: the search engine would find an action to be executed by the observer in order to induce a particular mental state in the observed agent. Success could be assessed by the model through observation of the other agents’ reaction. The neural correlates for this model are likely to be distributed and could include the posterior part of the superior temporal sulcus, the medial prefrontal cortex, the temporal poles, the premotor cortex and the cerebellum. To assess the plausibility of the model and test possible associations between particular neural structures and the components of the model, we review previous studies of the neural correlates of mentalising and some associated processes. no notspecified http://www.kyb.tuebingen.mpg.de/ published -69 15017 15422 4847 7 JS McDonald J Schultz Tübingen, Germany2006-03-00 172 9th Tübingen Perception Conference (TWK 2006) Previous studies (Atick and Redlich, Field, Webster and Miyahara) have investigated how the visual system could optimally represent the 1/f amplitude spectrum of natural images. Computational studies (Atick and Redlich, Field) suggest that the cortical representation ought to be a “whitened” version of the amplitude spectrum of natural images, i.e. spatial frequencies are equally represented despite the abundance of low spatial frequencies and dearth of high spatial frequencies in photographs of real world scenes. Webster and Miyahara showed that adaptation to natural images attenuates sensitivity to low spatial frequencies effectively supporting the computational evidence. We attempt to measure to what degree different spatial frequencies contribute to the percept of an image, in order to determine the extent of whitening of the input. To do this we adapted subjects briefly (250ms) to textures (4 x 4 degrees) of different spatial frequencies (1, 2, 4, 8, 16 cycles/degree, bandwidth 1.4 octaves—full width at half maximum). Then we measured the perceived contrast of 1/f textures in the adapted region of the visual field using the following procedure: After each interval of adaptation subjects judged whether the texture in the adapted region had a higher or lower contrast than that of the same texture in a non-adapted region. The contrast of the comparator texture (non-adapted) was changed after each time the subject made a judgement according to a 1 up 1 down staircase. We found that attenuation of perceived contrast, due to adaptation, is greatest when the adapting frequencies are at the peak of the contrast sensitivity function. It seems there is some “whitening”; however this is, at best, incomplete. no notspecified http://www.kyb.tuebingen.mpg.de/ published -172 15017 15422 3811 7 J Schultz F de Vignemont Washington, DC, USA2005-11-00 35th Annual Meeting of the Society for Neuroscience (Neuroscience 2005) Theory-of-Mind, or mentalising, is defined as a cognitive process used to understand other peoples' actions based on mental states. Two main theories of mentalising have been much discussed in recent years: Simulation Theory (e.g. Goldman 1993; Gallese and Goldman 1998), and Theory-Theory (e.g. Gopnik 1993). Models derived from these ideas and integrating neuroscience findings have been proposed since (e.g. Frith and Frith 1999; Blakemore and Decety 2001; Wolpert, Doya and Kawato 2003). Here we use the main idea from one of these models (Wolpert et al 2003) to propose a tentative model of mentalising based on action prediction and semantic representation. We also review a few neuroimaging studies of the processes involved in the model. no notspecified http://www.kyb.tuebingen.mpg.de//fileadmin/user_upload/files/publications/SfN2005poster_[0].pdf published 0 15017 15422 SchultzFWF2005 7 J Schultz KJ Friston DM Wolpert CD Frith A Coruña, Spain2005-08-00 62 28th European Conference on Visual Perception An essential, evolutionarily stable feature of brain function is the detection of animate entities, and one of the main cues to identify them is their movement. We developed a model of a simple interaction between two objects, in which we could control the percept of animacy by varying one parameter. The two disk-like objects moved along separate random trajectories but were also influenced by each other's positions, such that one object followed the other, in a parametrically controlled fashion. An increase of the correlation between the object's movements varied the amount of interactivity and animacy observers attributed to them. Control animations were only different from the experimental in terms of the interactivity level, but not in terms of object speed and separation. Twelve observers lying in a magnetic-resonance-imaging scanner had to rate the amount of interactivity and the overall speed of the objects in separate, subsequent tasks. Behavioural results showed a significant difference in interactivity ratings between experimental and control stimuli, but no difference in speed ratings, as expected. There was no response-time difference between the tasks. The fMRI data revealed that activation in the posterior superior temporal sulcus and gyrus (pSTS/pSTG) increased in relation to the degree of correlated motion between the two objects. This activation increase was not different when subjects performed an explicit or implicit task while observing these interacting objects. These data suggest that the pSTS and pSTG play a role in the automatic identification of animate entities, by responding directly to an objective movement characteristic inducing the percept of animacy, such as the amount of interactivity between two moving objects. These findings are consistent with literature showing that, in monkey and human, pSTS and pSTG respond to stimuli displaying biological motion. no notspecified http://www.kyb.tuebingen.mpg.de/ published -62 15017 15422 3188 7 J Schultz JN Ingram DM Wolpert CD Frith San Diego, CA, USA2004-10-00 34th Annual Meeting of the Society for Neuroscience (Neuroscience 2004) The cortex surrounding the posterior superior temporal sulcus of humans and monkeys is known to be activated during observation of biological movements, including human actions (1,2). In our event-related fMRI experiment, 12 healthy human volunteers were asked to discriminate between 2 versions of four different human actions on the basis of their movement kinematics. The difficulty of the task was influenced by the number of joints showing differences between the two movement versions. Clusters in the posterior superior temporal sulcus region in both hemispheres were the only brain regions whose activity varied inversely with the number of joints with significant differences between the two movement versions (clusters identified by SPM RFX analysis with 12 subjects thresholded at p<0.001 uncorrected, correlation with activation in left STS: R² = 0.96, right STS: R²= 0.94). Activity in the cluster identified in the right posterior superior temporal sulcus also showed a trend to correlate positively with participants’ performance (non significant, p=0.13). These results suggest that the cortex surrounding the posterior superior temporal sulcus participates in the extraction of kinematic information from observed biological movements, with activity increasing with task difficulty. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 3189 7 J Schultz K Friston H Imamizu C Frith Budapest, Hungary2004-06-00 e2122 10th Annual Meeting of the Organization for Human Brain Mapping (HBM 2004) Background Behavioural studies of children and adults show that goal-directedness is an important cue for the attribution of animacy to elements of the environment. In monkey and human, the cortex surrounding the superior temporal sulcus (STS) and gyrus (STG) is known to respond to biological motion and to intentional actions [1], and could thus participate in the detection of animate entities [2]. A growing number of neuroimaging studies also indicate that the STS appears to be involved in the representation of mental states. We reasoned that if the STS and STG are sensitive to intentional motion, activation in these structures would vary with the amount of perceived goal-directed motion. Further, if the STS is involved in the attribution of mental states, this structure might respond more when an observed goal-directed behaviour is directed by a representation of the target’s goal rather than a representation of the target’s position. Also, previous studies revealed attention effects on activation of the STS during processing of biological motion and of socially relevant characteristics of a face. Which level of processing in the STS is affected by attention is not yet clear. Methods We devised two fMRI experiments to study the response of the STS and of other brain regions to goal-directedness and the role of attention on this process. We presented healthy adult volunteers with animations of two round shapes moving in a seemingly animate way. In experiment 1, we parametrically varied the amount of goal-directed motion of the two abstract moving objects. Subjects had either to rate the amount of interaction between the moving objects or their speed, which was manipulated independently. In experiment 2, we manipulated the strategy used to reach the goal: agents either seemed to use knowledge of the goals attributed to the target object or to follow the target object. Stimuli were controlled for speed and quantity of movement, and eye movements were monitored in experiment 1. Results Increase in goal-directed behaviour parametrically increased activation in STS/STG and also in the medial occipital cortex and fusiform gyrus, even when subjects performed an incidental task. In experiment 2, watching agents trying to reach targets by using knowledge about the goals of the target object increased activation in the STS/STG, but only when subjects payed attention to the chaser’s strategy and not to the outcome of the chase. Conclusion We conclude that 1) the cortex in the STS/STG region responds to goal-directed behaviour independently of the task performed by the subject, and 2) it reponds more when a chase appears to be directed by a representation of the target’s goal rather than the target’s position, but only when subjects explicitly look for strategies of the chaser. This suggests that only higher levels of processing of socially relevant characteristics are under the influence of attention. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 3190 7 J Schultz KJ Friston H Imamizu CD Frith New Orleans, LA, USA2003-11-00 33rd Annual Meeting of the Society for Neuroscience (Neuroscience 2003) Goal-directed behaviour is an important cue for the attribution of animacy to elements of the environment, as has been repeatedly shown in behavioural studies of children and adults. In monkey and human, the superior temporal sulcus (STS) and gyrus (STG) are known to respond to stimuli displaying biological motion, and could thus participate in the detection of living entities. As goal-directedness is important for the attribution of animacy, we expected STS and STG to respond when objects appearing animate try to reach a goal. As the STS also appears to be involved in tasks involving mentalizing, it might also be sensitive to the way the agents reach their goal. In two fMRI experiments, we presented healthy adult volunteers with two agents (interacting, round shapes moving in a seemingly animate way) and varied the goal-directedness in their behaviour. In exp. 1, we parametrically increased the goal-directedness in the interaction of the agents, and in exp. 2 we varied the strategy used to reach the goal: agents either seemed to rely on mentalizing or only on physical cues. Stimuli were controlled for speed and quantity of movement. Increase in goal-directed behaviour parametrically increased activation in STS and STG, even when subjects performed an incidental task. In exp. 2, watching agents using a mentalizing strategy increased activation in the STS and STG; this was reduced when subjects performed an incidental task. We conclude that 1) the STS / STG region responds to goal-directed behaviour independently of the task performed by the subject, and 2) it reponds more when the goal-directed behaviour is apparently relying on mentalizing. This second activation increase seems to be only significant when subjects explicitly look for mental states in the observed behaviour. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 3191 7 J Schultz K Friston D Wolpert CD Frith New York, NY, USA2003-04-00 189 10th Annual Meeting of the Cognitive Neuroscience Society no notspecified http://www.kyb.tuebingen.mpg.de/ published -189 3192 7 J Schultz J Ingram D Wolpert C Frith 2002-06-00 8th International Conference on Functional Mapping of the Human Brain (HBM 2003) no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 3193 7 J Schultz OR Goodenough R Frackowiak CD Frith 2001-06-00 S473 NeuroImage In law theory, an important distinction is made between legal reasoning and a natural, intuitive sense of justice. We designed a simple word-based experiment to investigate with fMRI whether the neural correlates of making law-based or justice-ba\cd decisions are different. no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 3194 7 J Schultz R Boeck P Linder 1999-00-00 Translation and Stability of mRNA Meeting at the Palo Alto Institute of Molecular Medicine no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 3195 7 J Schultz J-P Hornung 1994-03-00 Annual Meeting of the Swiss Societies for Experimental Biology (USGEB) no notspecified http://www.kyb.tuebingen.mpg.de/ published 0 3065 15 JWR Schultz 2004-00-00 no notspecified published SchultzFdBK2012 10 J Schultz AL Fernandez Cruz S de la Rosa HH Bülthoff K Kaulard KimESBW2012 10 BR Kim J Esins J Schultz I Bülthoff C Wallraven 4092 10 J Schultz HH Bülthoff