Action recognition is sensitive to the identity of the actor

Recognizing who is carrying out an action is essential for successful human interaction. Ylva Ferstl and her colleagues from the Department of Prof. Bülthoff for Human Perception, Cognition and Action at the Max Planck Institute for Biological Cybernetics investigated the dependency between identity information and action related processes by testing the sensitivity of neural action recognition processes to clothing and facial identity information with a behavioral adaptation paradigm. The results show that action adaptation effects are in fact modulated by both clothing information and the actor's facial identity. The finding demonstrates that neural processes underlying action recognition are sensitive to identity information (including facial identity) and thereby not exclusively tuned to actions. The findings suggest that such response properties are useful to help humans in knowing who carried out an action.

1. Why were you interested in this topic?
I was interested in this topic, because I am generally interested in how our brain works, but especially in respect to higher cognition, such as social cognition. This is why I was drawn to a research group using Virtual Reality settings for investigating social behaviour. Our setting allowed us to provide a more natural environment of human interactions, while having complete control over the environment, i.e. over the virtual person our subjects were interacting with. We created a virtual environment in which our subjects interacted with a virtual person, and we manipulated the visual appearance of this person. That is, in different conditions we either changed only the head of the virtual person (leaving the rest of the body the same), or we only changed the person's clothes, or both. However, the visual appearance of the virtual person did not affect the task the subjects were instructed to do. With this, we found out that actions are perceived differently when we perceive different persons (so either a different head or both a different head and different clothes). This implies that attributing an action to a person does not happen post-hoc to make sense of what was observed, but rather the actor's identity becomes an integral part of understanding which action we are observing. The same exact motion can hence lead us to make a different judgment about the intended action depending on who we are seeing as the actor. This may also mean that we can never judge an action unbiasedly, our perception is inevitably influenced by our judgement of the actor. 

2. What should the average person take away from your study? 
What the average person should take away from our study is that action recognition is much more interlinked with who is performing said action than previously thought. Our study also helps explain how we can quickly understand who carried out an observed action, even when we are confronted with a crowd of people carrying out actions in our field of view. More specifically, when we see a motion being performed, we need to process this motion and decide both which action this was and who carried it out. Our results suggest that we do this by processing the motion linked with the actor from the start, so that this actor will influence our conclusion about the action. This way, the same exact motion can also lead to a different assessment of the action when carried out by different people. 

3. What is the added value of your study/paper for society?
Our study provides a better understanding of the workings behind our everyday social interactions and it helps us understand how we can so easily and quickly attribute an action to the correct actor. We learned that our perception of actions is tightly linked with who the actor is. Hence, identity of the actor is not just processed independently and later integrated into our understanding of the processed action. Rather, the identity becomes part of the action processing. This both tells us that actions can be perceived differently depending on the identity of the actor, and helps us understand how we can so readily understand who carried out the action. 

4. Are there any major caveats? What questions still need to be addressed?
There are no major caveats. Future studies might investigate if there are additional factors influencing action processing. 

5. Is there anything else you would like to add?
We carried out our study using an immersive virtual reality setup with life-sized avatars, meaning that we could closely mimic real-life social interactions. This allows us to draw conclusions for human behaviour beyond the laboratory. This is important when we try to understand higher-level cognition, such as social cognition in this case, because a highly unnatural environment will likely lead to behaviour dissimilar from natural behaviour. Our setup does not confine subjects' movements, as they might in a brain scanner, and they see their interaction partner in 3D and in natural size in front of them, rather than seeing a miniature version on a computer screen. This setup aims imitate natural conditions of observing another person carrying out an action towards them. 


Authors/Co-authors: 
Ylva Ferstl, Heinrich H. Bülthoff, Stephan de la Rosa