We present TELESIM, a modular and plug-and-play framework for direct teleoperation of a robotic arm using a digital twin as the interface between the user and the robotic system.
We tested TELESIM by performing a user survey with 37 participants on two different robots using two different control modalities: a virtual reality controller and a finger mapping hardware controller using different grasping systems. Users were asked to teleoperate the robot to pick and place 3 cubes in a tower and to repeat this task as many times as possible in 10 minutes, with only 5 minutes of training beforehand.
Our experimental results show that most users were able to succeed by building at least a tower of 3 cubes regardless of the control modality or robot used, demonstrating the user-friendliness of TELESIM.
The controllers (in the black dotted line)(1) can be any system that outputs a 3D pose, while our framework is depicted in the blue dotted line. It accepts the pose given by (1) to update the 3D pose of a cube in the digital twin. The robot then calculates a path to this cube in real-time, while avoiding collision with the world (4). Finally, TELESIM can be plugged into any robotic system (6) via a ROS2 robot controller (5)\, as shown in the red dotted line.
The Steam Index VR Headset is marked as (1) on the far left, which acts as the world’s origin. The Baxter robot on the left (2) is controlled by the Steam Index controller (5). In front of it, the UR3 is on the right (3), with the Yale OpenHand T42 gripper, controlled by the Senseglove and HTC Vive tracker (4) on the left side of the brown table. Additionally, in the upper right corner (7), a view of the starting setup of the task, which consists of 3 cubes in a triangle pattern, while on the brown table, the cubes are arranged in the goal configuration (6).
85% of the participants can build at least one tower in 10 minutes using Baxter and the VR controller. However, there is a steady decline for each of the following towers, with only 5% of the users able to build 8 towers. This is in direct comparison to the UR3, with slightly less than 50% of the population failing to build one tower and 5% managing to build 4, half as many towers for Baxter.
We thank the University of Glasgow for the use of the Baxter robot and the UR3 robot. We also thank the EPSCR for funding this research. Finally we thank all the participants who took part in our experiments.
@article{audonnet2023telesim,
title={TELESIM: A Modular and Plug-and-Play Framework for Robotic Arm Teleoperation using a Digital Twin},
author={Florent P Audonnet and Jonathan Grizou and Andrew Hamilton and Gerardo Aragon-Camarasa},
year={2023},
eprint={2309.10579},
archivePrefix={arXiv},
primaryClass={cs.RO}
}