Teleoperation is a very vast field that stretches over many different disciplines, including computer science, engineering, control theory, and many more. A teleoperation scenario can be defined when a device, e.g., a machine, robot manipulator or mobile robot, is being controlled by a human operator who is located in a different location than the device. Both parts are connected by a communication channel that transports the desired control input from the operator to the device and the sensor readings from the device back to the operator.
I am investigating teleoperation setups, where the operator controls a remote aircraft, namely a multirotor unmanned aerial vehicle (UAV) capable of vertical takeoff and landing (VTOL). The UAV is controlled using input devices that can also be found in regular planes and helicopters. Visuals typically include cockpit instruments and displays, plus live video streams delivered by one or more cameras attached to the UAV.
The separation of the operator from the aircraft adds additional challenges to the already demading task of controlling said aircraft. This is mainly due to the charateristics of the channel used between the operator and the UAV. Low overall channel capacity, latencies, noise, and jitter are the main factors that degrade the quality of controlling the UAV.
One reason for this can be the lack of information decreases the sense of "being there". This lack of situational awareness (SA) inevitably leads to a worsened performance in controlling the UAV and, in the worst case, the UAV to crash. Having said that, increasing the channel capacity or adding new sources of information should lead to a better SA and performance of the operator. My hypothesis is that presenting the motion of the UAV to the operator as feedback cue adds to the overall channel capacity. Therefore, the performance of the operator should increase.
However, it is not clear how the motion feedback needs to be defined in order to maximize the performance. In some cases motion feedback might even pose as a source of disturbance for the operator, worsening the performance.
In my PhD thesis I investigate the impact of motion feedback cues on operator performance. I hope that I can find initial evidence that suport my hypothesis that motion feedback helps increasing operator performance in some teleoperation scenarios.
Tele-operation of a remote vehicle is a challenging task, due to limitations of the remote sensors (e.g. weight, quality, power consumption), the transmission to the ground station (e.g. introduced noise and latency), and the display of this information to the operator (e.g. distortions, quality). These limitations may then lead to poor operator performance, increased workload, or even a loss of situational awareness. Usually, tele-operation setups do not provide motion feedback that lets the operator experience the motion of the remote vehicle. By providing motion feedback, the overall available information is increased, with the potential of increased operator performance. In tele-operation scenarios, the operator is located in a different location than the remote vehicle. This spatial decoupling grants a certain freedom in how the motion feedback can defined. Indeed, the motion perceived by the operator does not need to represent the motion of the vehicle. Instead the motion feedback could be defined to include information about the task. One example of task-related motion feedback is rotating the operator in roll depending on the lateral position error in a precision hover task.
First, find evidence for the effectiveness of motion feedback representing the motion of the remote vehicle (vehicle state motion feedback). Test the hypothesis of spatial decoupling that can be exploited to shape motion feedback in order to include task-related information. Finally, test the application of task-related motion feedback in a variety of tele-operation tasks.
Participants completed a series of tele-operation tasks while experiencing different types of motion feedback, see Figure 1. We measured the performance and the control activity of the participants completing the task. Performance is defined as the accumulated error when completing a task; control activity is the combination of stick deflection and stick deflection velocity. We found that vehicle state motion feedback improves operator performance . We argue that the increased performance is a result of increased disturbance rejection capabilities of the operator. This would be in line with results of vehicle simulation research . In addition, we found a significant reduction of accumulated error for task-related motion feedback conditions, see Figure 2. Exploring task-related motion feedback is a promising approach to provide optimal feedback to the operator with the potential of overcoming the challenges posed by tele-operation.
|Figure 1 Technical overview for a teleoperation setup where operators control a remote octorotor in a Tracking Room, from within the cabin of the CyberMotion Simulator (CMS).||Figure 2 Results of a teleoperation experiment where particpants experienced different feedback conditions, i.e. vehicle-state motion feedback, task-related motion feedback, and visual quality.|
|||J. Lächele, P. Pretto, J. Venrooij, H. H. Bülthoff (2014), Motion feedback improves performance in teleoperating UAVs AHS International 70th Annual Forum|
|||G. L. Ricard, R. V. Parrish (1984), Pilot differences and motion cuing effects on simulated helicopter hover Human Factors: The Journal of the Human Factors and Ergonomics Society 26 249-256.|
I have been working on a simulation environment called SwarmSimX. SwarmSimX is used within the Autonomous Robotics and Human-Machine Systems Group as a development and testing environment for a group of micro aerial vehicles (MAV). The focus of SwarmSimX is to provide a highly modular simulation framework that allows for simulating physical and visual properties of a virtual environment in real-time.
Below you can see a video of a quadcopter flying in the Multi-Agent-Lab (MAL) of the Max-Planck-Institute for Biological Cybernetics. The quadcopter is controlled by a program that uses the desired position and velocity of a recorded flight of a real quadcopter as control input.
|- 2003||High School (Abitur)|
|2003 - 2004||Civilian service|
|2004 - 2012||
Eberhard Karls-University of Tübingen
|10.2005 - 05.2006||Research Assistant "Lehrstuhl für Rechnernetze und Internet" (Chair for computer networks and internet)|
|Recording of lectures, seminars, and other presentations|
|06.2006 - 07.2008||Web developer and system administrator "Wellness Interaktiv GmbH"|
|Implementation and maintenance of a content management system and web shop. Administration of multiple email- and web-servers.|
|03.2008 - 12.2008||Student research project "Ultraschall-Kollisionsvermeidung für Quadrocopter" (Collisionavoidance for quadrotors using ultrasonic range finders)|
|Design and implementation of embedded hardware/software interfacing the sensors/quadrotor to be used in the collision avoidance algorithm.|
|08.2008 - 08.2011||Research Assistant "Max Planck Institute for biological Cybernetics"|
|Implementation of a communication interface for the CyberMotion Simulator (CMS). Design of visualization software for the CMS that was used in the design of new experiments. Design and implementation of physically realistic simulation software used for human-in-the-loop testing of multiple robot interaction.|
|09.2011 - 02.2012||Master Thesis (Diplomarbeit) "Development of a Real-Time Simulation Environment for multiple Robot Systems"|
Conference papers (7):
, , and (May-2016) Effects of vehicle- and task-related motion feedback on operator performance in teleoperation In: Leveraging Emerging Technologies for Future Capabilities, , 72nd American Helicopter Society International Annual Forum (AHS 2016), Curran, Red Hook, NY, USA, 3310-3316.
, , , and (September-2015) Novel approach for calculating motion feedback in teleoperation, 7th European Conference on Mobile Robots (ECMR 2015), IEEE, Piscataway, NJ, USA, 1-6.
, , and (May-2014) Motion Feedback Improves Performance in Teleoperating UAVs, 70th American Helicopter Society International Annual Forum (AHS 2014), Curran, Red Hook, NY, USA, 1777-1785.
, , , , , and (May-2013) Interactive Demo: Haptic Remote Control of Multiple UAVs with Autonomous Cohesive Behavior, ICRA 2013 Workshop Towards Fully Decentralized Multi-Robot Systems: Hardware, Software and Integration, 1-3.
, , and (May-2013) SwarmSimX and TeleKyb: Two ROS-integrated Software Frameworks for Single- and Multi-Robot Applications, ICRA 2013 Workshop Towards Fully Decentralized Multi-Robot Systems: Hardware, Software and Integration, 1-3.
, , and (November-2012) SwarmSimX: Real-time Simulation Environment for Multi-robot Systems In: Simulation, Modeling, and Programming for Autonomous Robots, , 3rd International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR 2012), Springer, Berlin, Germany, 375-387, Series: Lecture Notes in Computer Science ; 7628.
, , and (May-2010) Visual-Vestibular Feedback for Enhanced Situational Awareness in Teleoperation of UAVs, 66th American Helicopter Society International Annual Forum (AHS 2010), AHS International, Alexandria, VA, USA, 2809-2818.