2023
Wang, Ziwei; Fei, Haolin; Huang, Yanpei; Rouxel, Quentin; Xiao, Bo; Li, Zhibin; Burdet, Etienne
Learning to Assist Bimanual Teleoperation using Interval Type-2 Polynomial Fuzzy Inference Journal Article
In: IEEE Transactions on Cognitive and Developmental Systems, pp. 1–1, 2023, ISSN: 2379-8939, (Conference Name: IEEE Transactions on Cognitive and Developmental Systems).
Abstract | Links | BibTeX | Tags: Bimanual manipulation, Collaboration, Fuzzy sets, Gaussian process, Human-robot collaboration, IT2 polynomial fuzzy system, Robot kinematics, Robot learning, Robots, Task analysis, Trajectory, Uncertainty
@article{wang_learning_2023,
title = {Learning to Assist Bimanual Teleoperation using Interval Type-2 Polynomial Fuzzy Inference},
author = { Ziwei Wang and Haolin Fei and Yanpei Huang and Quentin Rouxel and Bo Xiao and Zhibin Li and Etienne Burdet},
doi = {10.1109/TCDS.2023.3272730},
issn = {2379-8939},
year = {2023},
date = {2023-01-01},
journal = {IEEE Transactions on Cognitive and Developmental Systems},
pages = {1–1},
abstract = {Assisting humans in collaborative tasks is a promising application for robots, however effective assistance remains challenging. In this paper, we propose a method for providing intuitive robotic assistance based on learning from human natural limb coordination. To encode coupling between multiple-limb motions, we use a novel interval type-2 (IT2) polynomial fuzzy inference for modeling trajectory adaptation. The associated polynomial coefficients are estimated using a modified recursive least-square with a dynamic forgetting factor. We propose to employ a Gaussian process to produce robust human motion predictions, and thus address the uncertainty and measurement noise of the system caused by interactive environments. Experimental results on two types of interaction tasks demonstrate the effectiveness of this approach, which achieves high accuracy in predicting assistive limb motion and enables humans to perform bimanual tasks using only one limb.},
note = {Conference Name: IEEE Transactions on Cognitive and Developmental Systems},
keywords = {Bimanual manipulation, Collaboration, Fuzzy sets, Gaussian process, Human-robot collaboration, IT2 polynomial fuzzy system, Robot kinematics, Robot learning, Robots, Task analysis, Trajectory, Uncertainty},
pubstate = {published},
tppubtype = {article}
}
Assisting humans in collaborative tasks is a promising application for robots, however effective assistance remains challenging. In this paper, we propose a method for providing intuitive robotic assistance based on learning from human natural limb coordination. To encode coupling between multiple-limb motions, we use a novel interval type-2 (IT2) polynomial fuzzy inference for modeling trajectory adaptation. The associated polynomial coefficients are estimated using a modified recursive least-square with a dynamic forgetting factor. We propose to employ a Gaussian process to produce robust human motion predictions, and thus address the uncertainty and measurement noise of the system caused by interactive environments. Experimental results on two types of interaction tasks demonstrate the effectiveness of this approach, which achieves high accuracy in predicting assistive limb motion and enables humans to perform bimanual tasks using only one limb.
2022
Allemang-Trivalle, Arnaud; Eden, Jonathan; Huang, Yanpei; Ivanova, Ekaterina; Burdet, Etienne
Comparison of human trimanual performance between independent and dependent multiple-limb training modes Proceedings Article
In: 2022 9th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), pp. 1–6, 2022, (ISSN: 2155-1782).
Abstract | Links | BibTeX | Tags: Biomechatronics, Dynamics, Navigation, Robot kinematics, Task analysis, Training, Virtual reality
@inproceedings{allemang-trivalle_comparison_2022,
title = {Comparison of human trimanual performance between independent and dependent multiple-limb training modes},
author = { Arnaud Allemang-Trivalle and Jonathan Eden and Yanpei Huang and Ekaterina Ivanova and Etienne Burdet},
doi = {10.1109/BioRob52689.2022.9925417},
year = {2022},
date = {2022-08-01},
booktitle = {2022 9th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob)},
pages = {1–6},
abstract = {Human movement augmentation with a third robotic hand can extend human capability allowing a single user to perform three-hand tasks that would typically require cooperation with other people. However, as trimanual control is not typical in everyday activities, it is still unknown how to train people to acquire this capability efficiently. We conducted an experimental study to evaluate two different trimanual training modes with 24 subjects. This investigated how the different modes impact the transfer of learning of the acquired trimanual capability to another task. Two groups of twelve subjects were each trained in virtual reality for five weeks using either independent or dependent trimanual task repetitions. The training was evaluated by comparing performance before and after training in a gamified trimanual task. The results show that both groups of subjects improved their trimanual capabilities after training. However, this improvement appeared independent of training scheme.},
note = {ISSN: 2155-1782},
keywords = {Biomechatronics, Dynamics, Navigation, Robot kinematics, Task analysis, Training, Virtual reality},
pubstate = {published},
tppubtype = {inproceedings}
}
Human movement augmentation with a third robotic hand can extend human capability allowing a single user to perform three-hand tasks that would typically require cooperation with other people. However, as trimanual control is not typical in everyday activities, it is still unknown how to train people to acquire this capability efficiently. We conducted an experimental study to evaluate two different trimanual training modes with 24 subjects. This investigated how the different modes impact the transfer of learning of the acquired trimanual capability to another task. Two groups of twelve subjects were each trained in virtual reality for five weeks using either independent or dependent trimanual task repetitions. The training was evaluated by comparing performance before and after training in a gamified trimanual task. The results show that both groups of subjects improved their trimanual capabilities after training. However, this improvement appeared independent of training scheme.
Allemang-Trivalle, Arnaud; Eden, Jonathan; Ivanova, Ekaterina; Huang, Yanpei; Burdet, Etienne
How long does it take to learn trimanual coordination? Proceedings Article
In: 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 211–216, 2022, (ISSN: 1944-9437).
Abstract | Links | BibTeX | Tags: Robot kinematics, Task analysis, Training, Virtual reality
@inproceedings{allemangtrivalle_how_2022,
title = {How long does it take to learn trimanual coordination?},
author = { Arnaud Allemang-Trivalle and Jonathan Eden and Ekaterina Ivanova and Yanpei Huang and Etienne Burdet},
doi = {10.1109/RO-MAN53752.2022.9900646},
year = {2022},
date = {2022-08-01},
urldate = {2022-08-01},
booktitle = {2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)},
pages = {211–216},
abstract = {Supernumerary robotic limbs can act as intelligent prostheses or augment the motion of healthy people to achieve actions which are not possible with only two natural hands. However, as trimanual control is not typical in everyday activities, it is still unknown how different training could influence its acquisition. We conducted an experimental study to evaluate the impact of different forms of trimanual action on training. Two groups of twelve subjects were each trained in virtual reality for five weeks using either a three independent goals task or one dependent goal task. The success of their training was then evaluated by comparing their task performance and motion characteristics between sessions. The results show that subjects dramatically improved their trimanual task performance as a result of training. However, while they showed improved motion efficiency and reduced workload for tasks with multiple independent goals with practice, no such improvement was observed when they trained with the one coordinated goal task.},
note = {ISSN: 1944-9437},
keywords = {Robot kinematics, Task analysis, Training, Virtual reality},
pubstate = {published},
tppubtype = {inproceedings}
}
Supernumerary robotic limbs can act as intelligent prostheses or augment the motion of healthy people to achieve actions which are not possible with only two natural hands. However, as trimanual control is not typical in everyday activities, it is still unknown how different training could influence its acquisition. We conducted an experimental study to evaluate the impact of different forms of trimanual action on training. Two groups of twelve subjects were each trained in virtual reality for five weeks using either a three independent goals task or one dependent goal task. The success of their training was then evaluated by comparing their task performance and motion characteristics between sessions. The results show that subjects dramatically improved their trimanual task performance as a result of training. However, while they showed improved motion efficiency and reduced workload for tasks with multiple independent goals with practice, no such improvement was observed when they trained with the one coordinated goal task.
Huang, Yanpei; Eden, Jonathan; Ivanova, Ekaterina; Burdet, Etienne
Human Performance of Three Hands in Unimanual, Bimanual and Trimanual Tasks Proceedings Article
In: 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 1493–1497, 2022, (ISSN: 2694-0604).
Abstract | Links | BibTeX | Tags: Biology, Motion control, Robot kinematics, Task analysis, Training, Virtual reality
@inproceedings{huang_human_2022,
title = {Human Performance of Three Hands in Unimanual, Bimanual and Trimanual Tasks},
author = { Yanpei Huang and Jonathan Eden and Ekaterina Ivanova and Etienne Burdet},
doi = {10.1109/EMBC48229.2022.9871248},
year = {2022},
date = {2022-07-01},
booktitle = {2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)},
pages = {1493–1497},
abstract = {Trimanual operation using a robotic supernumerary limb is a new and challenging mechanism for human operators that could enable a single user to perform tasks requiring more than two hands. Foot-controlled interfaces have previously proven able to be intuitively controlled, enabling simple tasks to be performed. However, the effect of going from unimanual to bimanual and then to trimanual tasks on subjects performance and coordination is not well understood. In this paper, unimanual, bimanual and trimanual teleoperation tasks were performed in a virtual reality scene to evaluate the impact of extending to trimanual actions. 15 participants were required to move their limbs together in a coordinated reaching activity. The results show that the addition of another hand resulted in an increase in operating time, where the time increased in going from unimanual to bimanual operation and then increased further when going from bimanual to trimanual. Moreover, the success rate for performing bimanual and trimanual tasks was strongly influenced by the subject's performance in ipsilateral hand-foot activities, where the ipsilateral combination had a lower success rate than contralateral limbs. The addition of a hand did not affect any two-hand coordination rate and even in some cases reduced coordination deviations. Clinical relevance - This work can contribute to build efficient training and learning framework on human multiple limbs motion control and coordination for both rehabilitation and augmentation.},
note = {ISSN: 2694-0604},
keywords = {Biology, Motion control, Robot kinematics, Task analysis, Training, Virtual reality},
pubstate = {published},
tppubtype = {inproceedings}
}
Trimanual operation using a robotic supernumerary limb is a new and challenging mechanism for human operators that could enable a single user to perform tasks requiring more than two hands. Foot-controlled interfaces have previously proven able to be intuitively controlled, enabling simple tasks to be performed. However, the effect of going from unimanual to bimanual and then to trimanual tasks on subjects performance and coordination is not well understood. In this paper, unimanual, bimanual and trimanual teleoperation tasks were performed in a virtual reality scene to evaluate the impact of extending to trimanual actions. 15 participants were required to move their limbs together in a coordinated reaching activity. The results show that the addition of another hand resulted in an increase in operating time, where the time increased in going from unimanual to bimanual operation and then increased further when going from bimanual to trimanual. Moreover, the success rate for performing bimanual and trimanual tasks was strongly influenced by the subject's performance in ipsilateral hand-foot activities, where the ipsilateral combination had a lower success rate than contralateral limbs. The addition of a hand did not affect any two-hand coordination rate and even in some cases reduced coordination deviations. Clinical relevance - This work can contribute to build efficient training and learning framework on human multiple limbs motion control and coordination for both rehabilitation and augmentation.
Huang, Yanpei; Ivanova, Ekaterina; Eden, Jonathan; Burdet, Etienne
Identification of Multiple Limbs Coordination Strategies in a Three-Goal Independent Task Journal Article
In: IEEE Transactions on Medical Robotics and Bionics, vol. 4, no. 2, pp. 348–351, 2022, ISSN: 2576-3202, (Conference Name: IEEE Transactions on Medical Robotics and Bionics).
Abstract | Links | BibTeX | Tags: Analysis of variance, foot control, Measurement, Random sequences, Robot kinematics, Robots, Surgery, Task analysis, teleoperation, three-hand surgery, Tri-manipulation
@article{huang_identification_2022,
title = {Identification of Multiple Limbs Coordination Strategies in a Three-Goal Independent Task},
author = { Yanpei Huang and Ekaterina Ivanova and Jonathan Eden and Etienne Burdet},
doi = {10.1109/TMRB.2021.3124263},
issn = {2576-3202},
year = {2022},
date = {2022-05-01},
journal = {IEEE Transactions on Medical Robotics and Bionics},
volume = {4},
number = {2},
pages = {348–351},
abstract = {Many surgical tasks require three or more tools operating together. A supernumerary robotic arm under the surgeon’s control could enable one surgeon to control three surgical tools simultaneously without assistance, thereby avoiding the common communication errors of the operation room. However, how do humans consider the complexity of controlling more than two arms together? In this paper, the coordination strategy used during three limb independent motion tasks is studied. The level of coordination increased over a two-day pilot study, and the resulting coordination pattern was in general consistent within subjects. Whether the subject used a fixed order of targets or a random sequence was found to reduce the improvement of pattern consistency after practice. The foot-controlled third hand exhibited less consistent patterns.},
note = {Conference Name: IEEE Transactions on Medical Robotics and Bionics},
keywords = {Analysis of variance, foot control, Measurement, Random sequences, Robot kinematics, Robots, Surgery, Task analysis, teleoperation, three-hand surgery, Tri-manipulation},
pubstate = {published},
tppubtype = {article}
}
Many surgical tasks require three or more tools operating together. A supernumerary robotic arm under the surgeon’s control could enable one surgeon to control three surgical tools simultaneously without assistance, thereby avoiding the common communication errors of the operation room. However, how do humans consider the complexity of controlling more than two arms together? In this paper, the coordination strategy used during three limb independent motion tasks is studied. The level of coordination increased over a two-day pilot study, and the resulting coordination pattern was in general consistent within subjects. Whether the subject used a fixed order of targets or a random sequence was found to reduce the improvement of pattern consistency after practice. The foot-controlled third hand exhibited less consistent patterns.
2021
Pinardi, Mattia; Raiano, Luigi; Noccaro, Alessia; Formica, Domenico; Pino, Giovanni Di
Cartesian Space Feedback for Real Time Tracking of a Supernumerary Robotic Limb: a Pilot Study Proceedings Article
In: 2021 10th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 889–892, 2021, (ISSN: 1948-3554).
Abstract | Links | BibTeX | Tags: Legged locomotion, Neural engineering, Position measurement, Real-time systems, Robot kinematics, Tracking, Vibrations
@inproceedings{pinardi_cartesian_2021b,
title = {Cartesian Space Feedback for Real Time Tracking of a Supernumerary Robotic Limb: a Pilot Study},
author = {Mattia Pinardi and Luigi Raiano and Alessia Noccaro and Domenico Formica and Giovanni {Di Pino}},
url = {https://ieeexplore.ieee.org/document/9441174},
doi = {10.1109/NER49283.2021.9441174},
year = {2021},
date = {2021-05-01},
urldate = {2021-05-01},
booktitle = {2021 10th International IEEE/EMBS Conference on Neural Engineering (NER)},
pages = {889–892},
abstract = {We present a system to provide the user with real time proprioceptive feedback regarding the state of a supernumerary robotic limb (SRL). The system converted the robot kinematics into a vibration amplitude-frequency value, using a custom electronic board. Four eccentric-motors placed on the leg delivered the vibrotactile pattern to subjects. We measured the accuracy in real-time tracking of the robot end-effector position and the delay from the robot movement onset. We tested four subjects in a preliminary study, and we found an average Position Error and a Delay of 0.084 ± 0.010 m and 1.169 ± 0.408 s respectively, which validated the feasibility of the presented setup. Increasing the learning phase duration should further improve subject performance. Additionally, the present platform could easily be employed to test the efficacy of different feedbacks (such as joint angles and torques) for real time tracking of SRL.},
note = {ISSN: 1948-3554},
keywords = {Legged locomotion, Neural engineering, Position measurement, Real-time systems, Robot kinematics, Tracking, Vibrations},
pubstate = {published},
tppubtype = {inproceedings}
}
We present a system to provide the user with real time proprioceptive feedback regarding the state of a supernumerary robotic limb (SRL). The system converted the robot kinematics into a vibration amplitude-frequency value, using a custom electronic board. Four eccentric-motors placed on the leg delivered the vibrotactile pattern to subjects. We measured the accuracy in real-time tracking of the robot end-effector position and the delay from the robot movement onset. We tested four subjects in a preliminary study, and we found an average Position Error and a Delay of 0.084 ± 0.010 m and 1.169 ± 0.408 s respectively, which validated the feasibility of the presented setup. Increasing the learning phase duration should further improve subject performance. Additionally, the present platform could easily be employed to test the efficacy of different feedbacks (such as joint angles and torques) for real time tracking of SRL.
Huang, Yanpei; Lai, Wenjie; Cao, Lin; Burdet, Etienne; Phee, Soo Jay
Design and Evaluation of a Foot-Controlled Robotic System for Endoscopic Surgery Journal Article
In: IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 2469–2476, 2021, ISSN: 2377-3766, (Conference Name: IEEE Robotics and Automation Letters).
Abstract | Links | BibTeX | Tags: Bending, Endoscope manipulation, Endoscopes, Foot, foot control, Instruments, Robot kinematics, robot-assisted surgery, Robots, Surgery, teleoperation
@article{huang_design_2021,
title = {Design and Evaluation of a Foot-Controlled Robotic System for Endoscopic Surgery},
author = { Yanpei Huang and Wenjie Lai and Lin Cao and Etienne Burdet and Soo Jay Phee},
url = {https://ieeexplore.ieee.org/document/9362198/},
doi = {10.1109/LRA.2021.3062009},
issn = {2377-3766},
year = {2021},
date = {2021-04-01},
urldate = {2022-06-02},
journal = {IEEE Robotics and Automation Letters},
volume = {6},
number = {2},
pages = {2469–2476},
abstract = {In traditional endoscopic surgery, the surgeon manipulating the endoscopic instruments is supported by an assistant controlling the endoscope, where their coordination may be affected by communication errors. To address this issue, we introduce a foot interface controlled robotic endoscopic system, enabling the surgeon to simultaneously operate the endoscope and instruments. The system consists of a foot interface using four degrees of freedom (DoFs) foot gestures and a robotic driving system for a commercial flexible endoscope. The proposed robotic system was validated in teleoperation experiments performed by ten subjects, where foot teleoperation was compared with hand teleoperation and direct control of the endoscope with two hands. The participants could successfully teleoperate the endoscope with foot teleoperation and exhibited 40% faster task completion as with direct control. They found both foot and hand teleoperations comfortable and intuitive. Although hand teleoperation enabled the best performance, only foot teleoperation allows simultaneous control of three instruments.},
note = {Conference Name: IEEE Robotics and Automation Letters},
keywords = {Bending, Endoscope manipulation, Endoscopes, Foot, foot control, Instruments, Robot kinematics, robot-assisted surgery, Robots, Surgery, teleoperation},
pubstate = {published},
tppubtype = {article}
}
In traditional endoscopic surgery, the surgeon manipulating the endoscopic instruments is supported by an assistant controlling the endoscope, where their coordination may be affected by communication errors. To address this issue, we introduce a foot interface controlled robotic endoscopic system, enabling the surgeon to simultaneously operate the endoscope and instruments. The system consists of a foot interface using four degrees of freedom (DoFs) foot gestures and a robotic driving system for a commercial flexible endoscope. The proposed robotic system was validated in teleoperation experiments performed by ten subjects, where foot teleoperation was compared with hand teleoperation and direct control of the endoscope with two hands. The participants could successfully teleoperate the endoscope with foot teleoperation and exhibited 40% faster task completion as with direct control. They found both foot and hand teleoperations comfortable and intuitive. Although hand teleoperation enabled the best performance, only foot teleoperation allows simultaneous control of three instruments.