Cybernetics Approach to Perception & Action

Cybernetics Approach to Perception & Action

In the CAPA group, we investigate human manual control behavior in order to increase our understanding of how humans use information perceived from their environment to generate control actions. This knowledge can be used to better support humans when performing control tasks, such as steering a vehicle.

Goals

The main two goals of our group are 1) to understand human behavior in manual control tasks and 2) to apply this knowledge to investigate novel approaches for human-machine interfaces and vehicle-augmentation that can make personal aviation more accessible to pilots with limited fight experience.

Methods

In our research, we develop and adapt system identification techniques to create reliable models of human feedforward behavior [1-3], human neuromuscular dynamics [4-7], control behavior in closed-loop control tasks and helicopter dynamics [8-9].

We use these identified models to develop control techniques that make manual control tasks safer and easier for pilots with limited fight experience. For example, we develop haptic support systems to train or support an operator while performing flying control tasks [10-11]. We also augment the dynamics of helicopters to achieve handling qualities considered suitable for minimally trained pilots [12-15].

We evaluate our new control approaches by performing human-in-the-loop experiments in fixed- and motion-base simulators. In these experiments, pilots with limited flight experience are instructed to perform specific closed-loop control tasks. From the collected data we can verify how haptic support and control augmentation can make helicopters easier to operate and how the required amount of training can be reduced. [16-18].The knowledge obtained from these experiments brings us a step closer to making Personal Aerial Vehicles (PAVs) a feasible transportation option.We evaluate our new control approaches by performing human-in-the-loop experiments in fixed- and motion-base simulators. In these experiments, pilots with limited flight experience are instructed to perform specific closed-loop control tasks. From the collected data we can verify how haptic support and control augmentation can make helicopters easier to operate and how the required amount of training can be reduced. [16-18].The knowledge obtained from these experiments brings us a step closer to making Personal Aerial Vehicles (PAVs) a feasible transportation option.

Results

We have designed and tested a new variable Force-Stiffness Haptic Feedback for teaching non expert pilots how to perform a 1 Degree of Freedom (DoF) disturbance rejection task with a control-loaded sidestick [16]. The variable Force-Stiffness Haptic Feedback is based on two design parameters:

  1. the amount of help given to the human operator (designed to provide a desired deflection of the control device similar to that provided by an expert pilot) and
  2. the authority given to the pilot for contrasting the provided haptic aid (obtained by changing the stiffness of the control device).
MPI CyberMotion Simulator.

By modifying these two parameters the student pilots are brought from an automated system to a fully manual system. In this way, student pilots can gradually learn how to perform the task without any haptic aid. The training method was tested in a human-in-the-loop experiment which showed that participants quickly achieved similar performance after the haptic aid was removed, thus indicating that they had indeed learned to control the vehicle. We are now planning new experiments in which we will apply the same approach for a task with more DoF. 
In the past few years, we have investigated how we can apply augmentation strategies to transform the difficult-to-control helicopter dynamics into handling qualities considered suitable for Personal Aerial Vehicles users [12-15]. The proposed augmentation strategies have been tested in simulation and in human-in-the-loop experiments performed on the MPI CyberMotion Simulator (Figure). Results have shown that after a very short training, pilots with limited flight experience are able to achieve performance comparable to those of an experienced helicopter pilot [17]. We have tested the implemented augmentation strategies also in presence of turbulences and poor visual conditions. Results have shown that the robust capabilities of the implemented controllers allow minimally trained pilots to achieve good performance levels while maintaining low workload levels [18].

We have recently developed a synthetic Haptic Helicopter Trainer (HHT) to teach minimally trained pilots how to perform a hover maneuver on a helicopter [19]. A human-in-the-loop experiment performed on a fixed-base simulator has shown that the training is effective and that about 100 minutes are enough for letting student pilots learn how to stabilize a helicopter in hover. A follow-up experiment has demonstrated that the student pilots trained with our HHT were able to perform an accurate hover even in a more realistic environment, simulated on our MPI CyberMotion Simulator [20]. This result showed that a Transfer-of-Training (ToT) is possible. To further test the possibility of achieving a positive ToT, a preliminary experiment has been performed with a few participants on the actual helicopter. The instructor’s evaluation of the participants trained with the HHT has been very positive. Based on these promising results, we are now planning to perform a rigorous experiment to measure the ToT to an actual helicopter achieved by using our HHT. An effective Haptic Helicopter Trainer on a fixed-base simulator could allow PAVs pilots to learn how to stabilize and safely land a heli-like PAV in case of augmentation failure in less than two hours. Furthermore, the developed haptic trainer could be considered a cheap, safe and time-saving tool for PAV as well as Helicopter pilot training. We have recently developed a synthetic Haptic Helicopter Trainer (HHT) to teach minimally trained pilots how to perform a hover maneuver on a helicopter [19]. A human-in-the-loop experiment performed on a fixed-base simulator has shown that the training is effective and that about 100 minutes are enough for letting student pilots learn how to stabilize a helicopter in hover. A follow-up experiment has demonstrated that the student pilots trained with our HHT were able to perform an accurate hover even in a more realistic environment, simulated on our MPI CyberMotion Simulator [20]. This result showed that a Transfer-of-Training (ToT) is possible. To further test the possibility of achieving a positive ToT, a preliminary experiment has been performed with a few participants on the actual helicopter. The instructor’s evaluation of the participants trained with the HHT has been very positive. Based on these promising results, we are now planning to perform a rigorous experiment to measure the ToT to an actual helicopter achieved by using our HHT. An effective Haptic Helicopter Trainer on a fixed-base simulator could allow PAVs pilots to learn how to stabilize and safely land a heli-like PAV in case of augmentation failure in less than two hours. Furthermore, the developed haptic trainer could be considered a cheap, safe and time-saving tool for PAV as well as Helicopter pilot training.

We are currently working on a collaborative project with the Motion Perception and Simulation (MPSim) group to replace the current classical Motion Cueing of the MPI Cyber Motion Simulator with a more realistic Motion Cueing, based on a Model Predictive Control (MPC) approach. We want the new Motion Cueing to work in real-time such that complex control tasks, like controlling a helicopter, could be reproduced in a very realistic way on a motion-base simulator. This novel MPC based Motion Cueing would allow us to investigate how accurate and realistic motion capabilities of a simulator can influence the ToT of student pilots to actual helicopters.We are currently working on a collaborative project with the Motion Perception and Simulation (MPSim) group to replace the current classical Motion Cueing of the MPI Cyber Motion Simulator with a more realistic Motion Cueing, based on a Model Predictive Control (MPC) approach. We want the new Motion Cueing to work in real-time such that complex control tasks, like controlling a helicopter, could be reproduced in a very realistic way on a motion-base simulator. This novel MPC based Motion Cueing would allow us to investigate how accurate and realistic motion capabilities of a simulator can influence the ToT of student pilots to actual helicopters.

References

  1. Drop FM, De Vries R, Mulder M and Bülthoff HH (August-2016) The Predictability of a Target Signal Affects Manual Feedforward Control, 13th IFAC/IFIP/IFORS/IEA Symposium on Analysis, Design, and Evaluation of Human-Machine Systems (HMS 2016), Elsevier, Frankfurt a.M., Germany, IFAC-PapersOnLine, 49(19), 177–182.
  2. Drop FM, Pool DM, Mulder M and Bülthoff HH (August-2016) Constraints in Identification of Multi-Loop Feedforward Human Control Models, 13th IFAC/IFIP/IFORS/IEA Symposium on Analysis, Design, and Evaluation of Human-Machine Systems (HMS 2016), Elsevier, Frankfurt a.M., Germany, IFAC-PapersOnLine, 49(19), 7-12.
  3. Drop FM, Pool DM, van Paassen MM, Mulder M and Bülthoff HH (September-2016) Objective Model Selection for Identifying the Human Feedforward Response in Manual Control IEEE Transactions on Cybernetics Epub ahead.Drop FM, Pool DM, van Paassen MM, Mulder M and Bülthoff HH (September-2016) Objective Model Selection for Identifying the Human Feedforward Response in Manual Control IEEE Transactions on Cybernetics Epub ahead.
  4. Olivari M, Nieuwenhuizen FM, Bülthoff HH and Pollini L (January-2015) Identifying Time-Varying Neuromuscular Response: Experimental Evaluation of a RLS-based Algorithm, AIAA Modeling and Simulation Technologies Conference 2015: Held at the SciTech Forum 2015, Curran, Red Hook, NY, USA, 284-298.Olivari M, Nieuwenhuizen FM, Bülthoff HH and Pollini L (January-2015) Identifying Time-Varying Neuromuscular Response: Experimental Evaluation of a RLS-based Algorithm, AIAA Modeling and Simulation Technologies Conference 2015: Held at the SciTech Forum 2015, Curran, Red Hook, NY, USA, 284-298.
  5. Olivari M, Nieuwenhuizen F, Venrooij J, Bülthoff HH and Pollini L (December-2015) Methods for Multiloop Identification of Visual and Neuromuscular Pilot Responses IEEE Transactions on Cybernetics 45(12) 2780 - 2791.Olivari M, Nieuwenhuizen F, Venrooij J, Bülthoff HH and Pollini L (December-2015) Methods for Multiloop Identification of Visual and Neuromuscular Pilot Responses IEEE Transactions on Cybernetics 45(12) 2780 - 2791.
  6. Olivari M, Nieuwenhuizen FM, Bülthoff HH and Pollini L (October-2015) Identifying Time-Varying Neuromuscular Response: a Recursive Least-Squares Algorithm with Pseudoinverse, IEEE International Conference on Systems, Man, and Cybernetics (SMC 2015), IEEE, Piscataway, NJ, USA, 3079-3085.Olivari M, Nieuwenhuizen FM, Bülthoff HH and Pollini L (October-2015) Identifying Time-Varying Neuromuscular Response: a Recursive Least-Squares Algorithm with Pseudoinverse, IEEE International Conference on Systems, Man, and Cybernetics (SMC 2015), IEEE, Piscataway, NJ, USA, 3079-3085.
  7. Olivari M, Venrooij J, Nieuwenhuizen FM, Pollini L and Bülthoff HH (January-2016) Identifying Time-Varying Pilot Responses: A Regularized Recursive Least-Squares Algorithm, AIAA Modeling and Simulation Technologies Conference: Held at the AIAA SciTech Forum 2016, Curran, Red Hook, NY, USA, 385-399.Olivari M, Venrooij J, Nieuwenhuizen FM, Pollini L and Bülthoff HH (January-2016) Identifying Time-Varying Pilot Responses: A Regularized Recursive Least-Squares Algorithm, AIAA Modeling and Simulation Technologies Conference: Held at the AIAA SciTech Forum 2016, Curran, Red Hook, NY, USA, 385-399.
  8. Geluardi S, Nieuwenhuizen FM, Pollini L and Bülthoff HH (October-2015) Data Collection for Developing a Dynamic Model of a Light Helicopter, 39th European Rotorcraft Forum (ERF 2013), Curran, Red Hook, NY, USA, 419-433.Geluardi S, Nieuwenhuizen FM, Pollini L and Bülthoff HH (October-2015) Data Collection for Developing a Dynamic Model of a Light Helicopter, 39th European Rotorcraft Forum (ERF 2013), Curran, Red Hook, NY, USA, 419-433.
  9. Geluardi S, Nieuwenhuizen FM, Venrooij J, Pollini L and Bülthoff HH (January-2018) Frequency Domain System Identification of a Robinson R44 in Hover, Journal of the American Helicopter Society 63(1) 1-18.Geluardi S, Nieuwenhuizen FM, Venrooij J, Pollini L and Bülthoff HH (January-2018) Frequency Domain System Identification of a Robinson R44 in Hover, Journal of the American Helicopter Society 63(1) 1-18.
  10. D'Intino G, Olivari M, Geluardi S, Venrooij J, Innocenti M, Bülthoff HH and Pollini L (October-2016) Evaluation of Haptic Support System for Training Purposes in a Tracking Task, IEEE International Conference on Systems, Man, and Cybernetics (SMC 2016), IEEE, Piscataway, NJ, USA, 002169-002174.D'Intino G, Olivari M, Geluardi S, Venrooij J, Innocenti M, Bülthoff HH and Pollini L (October-2016) Evaluation of Haptic Support System for Training Purposes in a Tracking Task, IEEE International Conference on Systems, Man, and Cybernetics (SMC 2016), IEEE, Piscataway, NJ, USA, 002169-002174.
  11. D'Intino G, Olivari M, Geluardi S, Venrooij J, Pollini L and Bülthoff HH (January-11-2017) Experimental evaluation of haptic support systems for learning a 2-DoF tracking task, AIAA Modeling and Simulation Technologies Conference: Held at the AIAA SciTech Forum 2017, 1-10.D'Intino G, Olivari M, Geluardi S, Venrooij J, Pollini L and Bülthoff HH (January-11-2017) Experimental evaluation of haptic support systems for learning a 2-DoF tracking task, AIAA Modeling and Simulation Technologies Conference: Held at the AIAA SciTech Forum 2017, 1-10.
  12. Geluardi S, Nieuwenhuizen FM, Pollini L and Bülthoff HH (May-6-2015) Augmented Systems for a Personal Aerial Vehicle Using a Civil Light Helicopter Model, 71st American Helicopter Society International Annual Forum (AHS 2015), Curran, Red Hook, NY, USA, 1428-1436.Geluardi S, Nieuwenhuizen FM, Pollini L and Bülthoff HH (May-6-2015) Augmented Systems for a Personal Aerial Vehicle Using a Civil Light Helicopter Model, 71st American Helicopter Society International Annual Forum (AHS 2015), Curran, Red Hook, NY, USA, 1428-1436.
  13. Picardi G, Geluardi S, Olivari M and Bülthoff HH (May-2016) L1-based Model Following Control of an Identified Helicopter Model in Hover In: Leveraging Emerging Technologies for Future Capabilities, , 72nd American Helicopter Society International Annual Forum (AHS 2016), Curran, Red Hook, NY, USA, 1770-1777.Picardi G, Geluardi S, Olivari M and Bülthoff HH (May-2016) L1-based Model Following Control of an Identified Helicopter Model in Hover In: Leveraging Emerging Technologies for Future Capabilities, , 72nd American Helicopter Society International Annual Forum (AHS 2016), Curran, Red Hook, NY, USA, 1770-1777.
  14. Gerboni CA, Venrooij J, Nieuwenhuizen FM, Joos A, Fichter W and Bülthoff HH (January-2016) Control Augmentation Strategies for Helicopters used as Personal Aerial Vehicles in Low-speed Regime, AIAA Modeling and Simulation Technologies Conference: Held at the AIAA SciTech Forum 2016, Curran, Red Hook, NY, USA, 1002-1012.Gerboni CA, Venrooij J, Nieuwenhuizen FM, Joos A, Fichter W and Bülthoff HH (January-2016) Control Augmentation Strategies for Helicopters used as Personal Aerial Vehicles in Low-speed Regime, AIAA Modeling and Simulation Technologies Conference: Held at the AIAA SciTech Forum 2016, Curran, Red Hook, NY, USA, 1002-1012.
  15. Gerboni CA, Geluardi S, Venrooij J, Joos A, Fichter W and Bülthoff HH (January-11-2017) Development of model-following control laws for helicopters to achieve personal aerial vehicle's handling qualities, AIAA Modeling and Simulation Technologies Conference: Held at the AIAA SciTech Forum 2017, 1-16.Gerboni CA, Geluardi S, Venrooij J, Joos A, Fichter W and Bülthoff HH (January-11-2017) Development of model-following control laws for helicopters to achieve personal aerial vehicle's handling qualities, AIAA Modeling and Simulation Technologies Conference: Held at the AIAA SciTech Forum 2017, 1-16.
  16. Bufalo F, Olivari M, Geluardi S, Gerboni CA, Pollini L and Bülthoff HH (October-2017) Variable Force-Stiffness Haptic Feedback for Learning a Disturbance Rejection Task, IEEE International Conference on Systems, Man, and Cybernetics (SMC 2017), IEEE, Piscataway, NJ, USA, 1517-1522.Bufalo F, Olivari M, Geluardi S, Gerboni CA, Pollini L and Bülthoff HH (October-2017) Variable Force-Stiffness Haptic Feedback for Learning a Disturbance Rejection Task, IEEE International Conference on Systems, Man, and Cybernetics (SMC 2017), IEEE, Piscataway, NJ, USA, 1517-1522.
  17. Geluardi S, Venrooij J, Olivari M, Bülthoff HH and Pollini L (October-2017) Transforming Civil Helicopters into Personal Aerial Vehicles: Modeling, Control, and Validation, Journal of Guidance, Control, and Dynamics 40(10) 2481-2495.Geluardi S, Venrooij J, Olivari M, Bülthoff HH and Pollini L (October-2017) Transforming Civil Helicopters into Personal Aerial Vehicles: Modeling, Control, and Validation, Journal of Guidance, Control, and Dynamics 40(10) 2481-2495.
  18. Gerboni CA, Geluardi S, Fichter W and Bülthoff HH (May-2017) Investigation and Evaluation of Control Design Requirements for Future Personal Aerial Vehicles, 73rd American Helicopter Society International Annual Forum (AHS 2017), 1-12.Gerboni CA, Geluardi S, Fichter W and Bülthoff HH (May-2017) Investigation and Evaluation of Control Design Requirements for Future Personal Aerial Vehicles, 73rd American Helicopter Society International Annual Forum (AHS 2017), 1-12.
  19. Fabbroni D, Geluardi S, Gerboni CA, Olivari M, D'Intino G, Pollini L and Bülthoff HH (May-2017) Design of a Haptic Helicopter Trainer for Inexperienced Pilots, 73rd American Helicopter Society International Annual Forum (AHS 2017), 1-12.Fabbroni D, Geluardi S, Gerboni CA, Olivari M, D'Intino G, Pollini L and Bülthoff HH (May-2017) Design of a Haptic Helicopter Trainer for Inexperienced Pilots, 73rd American Helicopter Society International Annual Forum (AHS 2017), 1-12.
  20. Fabbroni D, Geluardi S, Gerboni CA, Olivari M, Pollini L and Bülthoff HH (September-2017) Quasi-Transfer-of-Training of Helicopter Trainer from Fixed-Base to Motion-Base Simulator, 43rd European Rotorcraft Forum (ERF 2017), Milano Fabbroni D, Geluardi S, Gerboni CA, Olivari M, Pollini L and Bülthoff HH (September-2017) Quasi-Transfer-of-Training of Helicopter Trainer from Fixed-Base to Motion-Base Simulator, 43rd European Rotorcraft Forum (ERF 2017), Milano
Web-View
Print Page
Open in new window
Estimated DIN-A4 page-width
Go to Editor View