2023
Pinardi, Mattia; Noccaro, Alessia; Raiano, Luigi; Formica, Domenico; Pino, Giovanni Di
Comparing end-effector position and joint angle feedback for online robotic limb tracking Journal Article
In: PLOS ONE, vol. 18, no. 6, pp. e0286566, 2023, ISSN: 1932-6203, (Publisher: Public Library of Science).
Abstract | Links | BibTeX | Tags: Body limbs, Motion, Prosthetics, Robotics, Robots, Sensory perception, Vibration, Vision
@article{pinardi_comparing_2023,
title = {Comparing end-effector position and joint angle feedback for online robotic limb tracking},
author = { Mattia Pinardi and Alessia Noccaro and Luigi Raiano and Domenico Formica and Giovanni {Di Pino}},
url = {https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0286566},
doi = {10.1371/journal.pone.0286566},
issn = {1932-6203},
year = {2023},
date = {2023-06-01},
urldate = {2023-06-01},
journal = {PLOS ONE},
volume = {18},
number = {6},
pages = {e0286566},
abstract = {Somatosensation greatly increases the ability to control our natural body. This suggests that supplementing vision with haptic sensory feedback would also be helpful when a user aims at controlling a robotic arm proficiently. However, whether the position of the robot and its continuous update should be coded in a extrinsic or intrinsic reference frame is not known. Here we compared two different supplementary feedback contents concerning the status of a robotic limb in 2-DoFs configuration: one encoding the Cartesian coordinates of the end-effector of the robotic arm (i.e., Task-space feedback) and another and encoding the robot joints angles (i.e., Joint-space feedback). Feedback was delivered to blindfolded participants through vibrotactile stimulation applied on participants’ leg. After a 1.5-hour training with both feedbacks, participants were significantly more accurate with Task compared to Joint-space feedback, as shown by lower position and aiming errors, albeit not faster (i.e., similar onset delay). However, learning index during training was significantly higher in Joint space feedback compared to Task-space feedback. These results suggest that Task-space feedback is probably more intuitive and more suited for activities which require short training sessions, while Joint space feedback showed potential for long-term improvement. We speculate that the latter, despite performing worse in the present work, might be ultimately more suited for applications requiring long training, such as the control of supernumerary robotic limbs for surgical robotics, heavy industrial manufacturing, or more generally, in the context of human movement augmentation.},
note = {Publisher: Public Library of Science},
keywords = {Body limbs, Motion, Prosthetics, Robotics, Robots, Sensory perception, Vibration, Vision},
pubstate = {published},
tppubtype = {article}
}
Somatosensation greatly increases the ability to control our natural body. This suggests that supplementing vision with haptic sensory feedback would also be helpful when a user aims at controlling a robotic arm proficiently. However, whether the position of the robot and its continuous update should be coded in a extrinsic or intrinsic reference frame is not known. Here we compared two different supplementary feedback contents concerning the status of a robotic limb in 2-DoFs configuration: one encoding the Cartesian coordinates of the end-effector of the robotic arm (i.e., Task-space feedback) and another and encoding the robot joints angles (i.e., Joint-space feedback). Feedback was delivered to blindfolded participants through vibrotactile stimulation applied on participants’ leg. After a 1.5-hour training with both feedbacks, participants were significantly more accurate with Task compared to Joint-space feedback, as shown by lower position and aiming errors, albeit not faster (i.e., similar onset delay). However, learning index during training was significantly higher in Joint space feedback compared to Task-space feedback. These results suggest that Task-space feedback is probably more intuitive and more suited for activities which require short training sessions, while Joint space feedback showed potential for long-term improvement. We speculate that the latter, despite performing worse in the present work, might be ultimately more suited for applications requiring long training, such as the control of supernumerary robotic limbs for surgical robotics, heavy industrial manufacturing, or more generally, in the context of human movement augmentation.
Wang, Ziwei; Fei, Haolin; Huang, Yanpei; Rouxel, Quentin; Xiao, Bo; Li, Zhibin; Burdet, Etienne
Learning to Assist Bimanual Teleoperation using Interval Type-2 Polynomial Fuzzy Inference Journal Article
In: IEEE Transactions on Cognitive and Developmental Systems, pp. 1–1, 2023, ISSN: 2379-8939, (Conference Name: IEEE Transactions on Cognitive and Developmental Systems).
Abstract | Links | BibTeX | Tags: Bimanual manipulation, Collaboration, Fuzzy sets, Gaussian process, Human-robot collaboration, IT2 polynomial fuzzy system, Robot kinematics, Robot learning, Robots, Task analysis, Trajectory, Uncertainty
@article{wang_learning_2023,
title = {Learning to Assist Bimanual Teleoperation using Interval Type-2 Polynomial Fuzzy Inference},
author = { Ziwei Wang and Haolin Fei and Yanpei Huang and Quentin Rouxel and Bo Xiao and Zhibin Li and Etienne Burdet},
doi = {10.1109/TCDS.2023.3272730},
issn = {2379-8939},
year = {2023},
date = {2023-01-01},
journal = {IEEE Transactions on Cognitive and Developmental Systems},
pages = {1–1},
abstract = {Assisting humans in collaborative tasks is a promising application for robots, however effective assistance remains challenging. In this paper, we propose a method for providing intuitive robotic assistance based on learning from human natural limb coordination. To encode coupling between multiple-limb motions, we use a novel interval type-2 (IT2) polynomial fuzzy inference for modeling trajectory adaptation. The associated polynomial coefficients are estimated using a modified recursive least-square with a dynamic forgetting factor. We propose to employ a Gaussian process to produce robust human motion predictions, and thus address the uncertainty and measurement noise of the system caused by interactive environments. Experimental results on two types of interaction tasks demonstrate the effectiveness of this approach, which achieves high accuracy in predicting assistive limb motion and enables humans to perform bimanual tasks using only one limb.},
note = {Conference Name: IEEE Transactions on Cognitive and Developmental Systems},
keywords = {Bimanual manipulation, Collaboration, Fuzzy sets, Gaussian process, Human-robot collaboration, IT2 polynomial fuzzy system, Robot kinematics, Robot learning, Robots, Task analysis, Trajectory, Uncertainty},
pubstate = {published},
tppubtype = {article}
}
Assisting humans in collaborative tasks is a promising application for robots, however effective assistance remains challenging. In this paper, we propose a method for providing intuitive robotic assistance based on learning from human natural limb coordination. To encode coupling between multiple-limb motions, we use a novel interval type-2 (IT2) polynomial fuzzy inference for modeling trajectory adaptation. The associated polynomial coefficients are estimated using a modified recursive least-square with a dynamic forgetting factor. We propose to employ a Gaussian process to produce robust human motion predictions, and thus address the uncertainty and measurement noise of the system caused by interactive environments. Experimental results on two types of interaction tasks demonstrate the effectiveness of this approach, which achieves high accuracy in predicting assistive limb motion and enables humans to perform bimanual tasks using only one limb.
2022
Peña-Pérez, Nuria; Eden, Jonathan; Ivanova, Ekaterina; Burdet, Etienne; Farkhatdinov, Ildar
Is a Robot Needed to Modify Human Effort in Bimanual Tracking? Journal Article
In: IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 8069–8075, 2022, ISSN: 2377-3766, (Conference Name: IEEE Robotics and Automation Letters).
Abstract | Links | BibTeX | Tags: design and human factors, Haptic interfaces, human-centered robo- tics, Perturbation methods, Rehabilitation robotics, Robots, Task analysis, Training, Visualization, Wrist
@article{pena_perez_is_2022,
title = {Is a Robot Needed to Modify Human Effort in Bimanual Tracking?},
author = {Nuria Peña-Pérez and Jonathan Eden and Ekaterina Ivanova and Etienne Burdet and Ildar Farkhatdinov},
doi = {10.1109/LRA.2022.3183753},
issn = {2377-3766},
year = {2022},
date = {2022-07-01},
urldate = {2022-07-01},
journal = {IEEE Robotics and Automation Letters},
volume = {7},
number = {3},
pages = {8069–8075},
abstract = {Robotic bimanual training can benefit from understanding how to modify human motor effort in bimanual tasks. We addressed this issue by carrying out a study to investigate whether and how penalizing the use of one hand could alter the hands’ effort distribution. Actuated haptic perturbations and alterations of the visual feedback of the right hand were tested on a bimanual tracking task with 16 healthy right-handed participants. For each feedback modality (haptic or visual), both a disturbance and a perturbation requiring additional effort from the right hand were implemented. The results showed that the participants were able to adjust to these four perturbations, and perceived them correctly as something that disturbed the dominant hand. Contrary to our expectations, the bimanual effort distribution changes induced by the haptic perturbations were not uniform across subjects. However, the visual disturbance induced most participants to use only their unperturbed left hand (with only 2/16 participants reporting a different behaviour). This suggests that a visual disturbance could be used to alter the effort distribution among the two hands. Clinical validation of these findings on hemiplegic patients may help simplify the design of robotic training interfaces.},
note = {Conference Name: IEEE Robotics and Automation Letters},
keywords = {design and human factors, Haptic interfaces, human-centered robo- tics, Perturbation methods, Rehabilitation robotics, Robots, Task analysis, Training, Visualization, Wrist},
pubstate = {published},
tppubtype = {article}
}
Robotic bimanual training can benefit from understanding how to modify human motor effort in bimanual tasks. We addressed this issue by carrying out a study to investigate whether and how penalizing the use of one hand could alter the hands’ effort distribution. Actuated haptic perturbations and alterations of the visual feedback of the right hand were tested on a bimanual tracking task with 16 healthy right-handed participants. For each feedback modality (haptic or visual), both a disturbance and a perturbation requiring additional effort from the right hand were implemented. The results showed that the participants were able to adjust to these four perturbations, and perceived them correctly as something that disturbed the dominant hand. Contrary to our expectations, the bimanual effort distribution changes induced by the haptic perturbations were not uniform across subjects. However, the visual disturbance induced most participants to use only their unperturbed left hand (with only 2/16 participants reporting a different behaviour). This suggests that a visual disturbance could be used to alter the effort distribution among the two hands. Clinical validation of these findings on hemiplegic patients may help simplify the design of robotic training interfaces.
Huang, Yanpei; Ivanova, Ekaterina; Eden, Jonathan; Burdet, Etienne
Identification of Multiple Limbs Coordination Strategies in a Three-Goal Independent Task Journal Article
In: IEEE Transactions on Medical Robotics and Bionics, vol. 4, no. 2, pp. 348–351, 2022, ISSN: 2576-3202, (Conference Name: IEEE Transactions on Medical Robotics and Bionics).
Abstract | Links | BibTeX | Tags: Analysis of variance, foot control, Measurement, Random sequences, Robot kinematics, Robots, Surgery, Task analysis, teleoperation, three-hand surgery, Tri-manipulation
@article{huang_identification_2022,
title = {Identification of Multiple Limbs Coordination Strategies in a Three-Goal Independent Task},
author = { Yanpei Huang and Ekaterina Ivanova and Jonathan Eden and Etienne Burdet},
doi = {10.1109/TMRB.2021.3124263},
issn = {2576-3202},
year = {2022},
date = {2022-05-01},
journal = {IEEE Transactions on Medical Robotics and Bionics},
volume = {4},
number = {2},
pages = {348–351},
abstract = {Many surgical tasks require three or more tools operating together. A supernumerary robotic arm under the surgeon’s control could enable one surgeon to control three surgical tools simultaneously without assistance, thereby avoiding the common communication errors of the operation room. However, how do humans consider the complexity of controlling more than two arms together? In this paper, the coordination strategy used during three limb independent motion tasks is studied. The level of coordination increased over a two-day pilot study, and the resulting coordination pattern was in general consistent within subjects. Whether the subject used a fixed order of targets or a random sequence was found to reduce the improvement of pattern consistency after practice. The foot-controlled third hand exhibited less consistent patterns.},
note = {Conference Name: IEEE Transactions on Medical Robotics and Bionics},
keywords = {Analysis of variance, foot control, Measurement, Random sequences, Robot kinematics, Robots, Surgery, Task analysis, teleoperation, three-hand surgery, Tri-manipulation},
pubstate = {published},
tppubtype = {article}
}
Many surgical tasks require three or more tools operating together. A supernumerary robotic arm under the surgeon’s control could enable one surgeon to control three surgical tools simultaneously without assistance, thereby avoiding the common communication errors of the operation room. However, how do humans consider the complexity of controlling more than two arms together? In this paper, the coordination strategy used during three limb independent motion tasks is studied. The level of coordination increased over a two-day pilot study, and the resulting coordination pattern was in general consistent within subjects. Whether the subject used a fixed order of targets or a random sequence was found to reduce the improvement of pattern consistency after practice. The foot-controlled third hand exhibited less consistent patterns.
2021
Huang, Yanpei; Lai, Wenjie; Cao, Lin; Burdet, Etienne; Phee, Soo Jay
Design and Evaluation of a Foot-Controlled Robotic System for Endoscopic Surgery Journal Article
In: IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 2469–2476, 2021, ISSN: 2377-3766, (Conference Name: IEEE Robotics and Automation Letters).
Abstract | Links | BibTeX | Tags: Bending, Endoscope manipulation, Endoscopes, Foot, foot control, Instruments, Robot kinematics, robot-assisted surgery, Robots, Surgery, teleoperation
@article{huang_design_2021,
title = {Design and Evaluation of a Foot-Controlled Robotic System for Endoscopic Surgery},
author = { Yanpei Huang and Wenjie Lai and Lin Cao and Etienne Burdet and Soo Jay Phee},
url = {https://ieeexplore.ieee.org/document/9362198/},
doi = {10.1109/LRA.2021.3062009},
issn = {2377-3766},
year = {2021},
date = {2021-04-01},
urldate = {2022-06-02},
journal = {IEEE Robotics and Automation Letters},
volume = {6},
number = {2},
pages = {2469–2476},
abstract = {In traditional endoscopic surgery, the surgeon manipulating the endoscopic instruments is supported by an assistant controlling the endoscope, where their coordination may be affected by communication errors. To address this issue, we introduce a foot interface controlled robotic endoscopic system, enabling the surgeon to simultaneously operate the endoscope and instruments. The system consists of a foot interface using four degrees of freedom (DoFs) foot gestures and a robotic driving system for a commercial flexible endoscope. The proposed robotic system was validated in teleoperation experiments performed by ten subjects, where foot teleoperation was compared with hand teleoperation and direct control of the endoscope with two hands. The participants could successfully teleoperate the endoscope with foot teleoperation and exhibited 40% faster task completion as with direct control. They found both foot and hand teleoperations comfortable and intuitive. Although hand teleoperation enabled the best performance, only foot teleoperation allows simultaneous control of three instruments.},
note = {Conference Name: IEEE Robotics and Automation Letters},
keywords = {Bending, Endoscope manipulation, Endoscopes, Foot, foot control, Instruments, Robot kinematics, robot-assisted surgery, Robots, Surgery, teleoperation},
pubstate = {published},
tppubtype = {article}
}
In traditional endoscopic surgery, the surgeon manipulating the endoscopic instruments is supported by an assistant controlling the endoscope, where their coordination may be affected by communication errors. To address this issue, we introduce a foot interface controlled robotic endoscopic system, enabling the surgeon to simultaneously operate the endoscope and instruments. The system consists of a foot interface using four degrees of freedom (DoFs) foot gestures and a robotic driving system for a commercial flexible endoscope. The proposed robotic system was validated in teleoperation experiments performed by ten subjects, where foot teleoperation was compared with hand teleoperation and direct control of the endoscope with two hands. The participants could successfully teleoperate the endoscope with foot teleoperation and exhibited 40% faster task completion as with direct control. They found both foot and hand teleoperations comfortable and intuitive. Although hand teleoperation enabled the best performance, only foot teleoperation allows simultaneous control of three instruments.