Tim Genewein

Alumni of the Research Group Sensorimotor Learning and Decision-Making

Main Focus

  • Find more information about me on

Research Group:

Biological organisms as well as artificial agents are faced with the problem of coping with a persistently varying environment. The process of learning in such an environment is twofold:

  • adaptation to the current parameters of the environment
  • extraction of invariants across a variety of environmental parameters

The latter learning process is referred to as structural learning which requires the formation of abstract representations, potentially on multiple levels of abstraction. From an information theoretic point of view, abstractions are compressed representations that carry relevant information and neglect irrelevant information – e.g. the abstract concept of a chair, where particular features like color, material or weight are considered irrelevant.The corresponding information theoretic framework for lossy compression is called rate-distortion theory.

Using the rate-distortion formalism we recently showed how a limit in information processing capacity leads to the emergence of abstractions in artificial agents that maximize their expected utility or reward . Importantly, the granularity of the abstraction depends directly on the cost of information processing.

Currently, we are working on extending this idea to modelling a hierarchy of several layers of abstraction. In current models of hierarchies of abstractions such as hierarchical Bayesian models or Connectionist models and deep networks, information processing capabilities are implicitly constrained by the model architecture but are not explicitly considered in their analysis. We argue that abstractions arise as a consequence of information processing limits and that the emergence of abstractions is heavily guided by information processing cost. We thus propose to explore an information theoretic point of view on the formation of abstract representations.

  • More about my current research is on

Introduction


A central question in the computational study of biological motor control is the investigation of principles and mechanisms that underlie the nervous system’s ability to learn new sensorimotor transformations [1]. This comprises not only the ability to learn new mappings from sensory stimuli to motor commands, but also the ability to predict sensory consequences of one’s own actions. Such sensorimotor transformations are thought to be represented in the nervous system as internal models. A forward internal model predicts the causal relationship between actions and their sensory consequences, while sometimes the phrase inverse internal model is used to indicate the mapping from a desired sensory state to the appropriate motor command. While forward models can easily be acquired by self-supervised learning, the acquisition of an inverse model or control law is computationally much more difficult. Recent studies have shown that humans make use of internal models [2].
From a theoretical point of view there are two levels of analysis with regard to the problem of learning how to act (the adaptive control problem): structural adaptive control and parametric adaptive control. In parametric adaptive control, knowledge of the structure of the control task is presupposed, and only unknown parameters of a particular task environment of that class have to be estimated. Conversely, when the structure of the adaptive control problem is unknown, the relevant control variables have to be extracted first as well as their interrelations, dimensionality, range of potential values, time scales, stochasticity etc. The problem of structural motor learning for a class of similar motor tasks is related to the problem of transfer learning or inductive learning that is investigated in machine learning.

Project A: How do humans select among different structures?
This project investigates the question of how the motor system selects between different structures or models, when each model can be associated with a range of different task-specific parameters. We design a sensorimotor task that requires subjects to compensate visuomotor shifts in a three-dimensional virtual reality setup, where one of the dimensions can be mapped to a model variable and the other dimension to the parameter variable. By introducing probe trials that are neutral in the parameter dimension, we can directly test for model selection. Our experimental design lends itself to the general study of hierarchical Bayesian inference in a sensorimotor context that requires the integration of subjects’ prior beliefs over different structures. Thus, we can test whether subjects prefer “simple” structures when presented with ambiguous evidence in line with Occam’s razor (i.e. Bayesian model comparison based on Bayes’ factors).

Project B: Information theoretic foundations of structure learning.

Learning structure is the problem of separating invariant structural information from variations of specific instances of the structure - in other words: separating structure from noise. Consider the abstract concept of a chair, where large parts of the sensory information induced by a specific chair, like color, material or weight are considered as variations or noise, whereas other parts of the sensory information are absolutely crucial. The abstract chair can be viewed as a (lossy) compression of the sensory information.

The information theoretic framework for analyzing lossy compression is rate-distortion theory where the problem lies in transmitting information over a channel with limited capacity, leading to the necessity to discard the most irrelevant information. Essentially this is the same problem as separating (relevant) structure from (irrelevant) noise. As we show in [3], the mathematical formalism of rate-distortion theory can also be applied to an agent with limited computational capacity that maximizes its utility, leading to the emergence of abstractions. Importantly these abstractions are induced by having limited information processing capabilities and the granularity of the abstractions is directly influenced by the cost of information processing.

The work in [3] and the close relation to the thermodynamic framework for decision-making with information processing cost lead to the following conclusions:

  • A limit in information processing capacity is equivalent to having a certain cost of information processing.
  • Abstractions (or a certain kind of robustness) arise when trading of maximum expected utility with information processing cost.
  • Abstractions emerge as a consequence of limited information processing capacity.

We plan on extending our work to modelling several layers of abstractions. Commonly used models for modelling hierarchies of abstractions such as hierarchical Bayesian models or Connectionist models and deep networks implicitly set an information processing limit through their architecture but this is usually not considered in the analysis. Our previous results suggest that information processing cost plays a crucial role in the emergence of abstractions and thus furhter exploration of the information theoretic point of view on the formation of abstractions seems promising. Potentially this could also add a novel theoretical point of view on existing models of hierarchies of abstractions.

References

[1] D. M. Wolpert and Z. Ghahramani, "Computational principles of movement neuroscience", Nat Neurosci, 2000

[2] D. M. Wolpert, Z. Ghahramani, and M. I. Jordan, "An internal model for sensorimotor integration", Science, 1995

[3] T. Genewein and D. A. Braun, "Abstraction in decision-makers with limited information processing capabilities", NIPS 2013 Workshop on Planning with Information Constraints

Curriculum Vitae

Education

  • 2012 - Jan.: Joined the Sensorimotor Learning and Decision Making group as a PhD candidate.
  • 2012 - Mar.: MSc Telematics, Graz University of Technology, Austria
    • Thesis: Structure Learning for Robotic Motor Control
  • 2009 - Oct.: BSc Telematics, Graz University of Technology, Austria

Work Experience

  • 2010 to 2011: Software developer (part-time) at NTE Systems, Graz
  • 2009 to 2010: Software developer (part-time) at IVM Engineering, Graz
Go to Editor View