Online Learning of Dynamic Control for Soft Manipulators

Soft robots offer notable advantages over rigid robots in terms of flexibility and compliance, which facilitate safe and robust interactions with environments. These characteristics make them ideal for diverse applications, such as medical contexts and bionic robotics. However, due to the absence of precise models and the time-varying nature of their system dynamics, achieving optimal control performance on soft robots remains a significant challenge. With the rapid development of machine learning, we now have various learning-based methods to achieve precise pose control of soft robots [4]. Nevertheless, in practice, we have found that the (off-line) well-trained control policies often become ineffective upon deployment due to time-varying system dynamics, which lead us to rethink the traditional “sampling-training-deployment” paradigm in machine learning. The rise of online learning can effectively enable models to adapt to changing system dynamics. Recent studies have shown that in online learning, even when using an approximate (or even linear) model, it is possible to ensure a sublinear convergence of the regret [2]. Additionally, online learning demonstrates high learning efficiency and stability in both simulations and complex real-world systems. In this project, we will adopt gradient-based stochastic online learning to achieve precise pose control of a soft manipulator [1,3]. The cable-driven soft robot arm comprises three independent modules (about 0.2m for each module), and each module is actuated independently with three cables. More details of the robot setup, such as structural and mechanical description, can be found in our previous work [3]. We will first establish an approximate model of the system, then learn both feedforward and feedback controllers in an online manner, which enable the soft arm to perform a series of acrobatic demonstrations. Finally, we will compare our approach with some existing offline control algorithms to evaluate the performance in terms of learning efficiency, handling distribution shifts, and tracking accuracy. Reference [1] Chen, Z., Guan, Q., Hughes, J., Menciassi, A. and Stefanini, C., 2024. S2C2A: A Flexible Task Space Planning and Control Strategy for Modular Soft Robot Arms. arXiv preprint arXiv:2410.03483. [2] Ma, H., Zeilinger, M. and Muehlebach, M., 2024. Stochastic Online Optimization for Cyber-Physical and Robotic Systems. arXiv preprint arXiv:2404.05318. [3] Guan, Q., Stella, F., Della Santina, C., Leng, J. and Hughes, J., 2023. Trimmed Helicoids: An Architectured Soft Structure yielding Soft Robots with High Precision, Large Workspace, and Compliant Interactions. npj Robotics, 1(1), p. 4. [4] Bern, J.M., Schnider, Y., Banzet, P., Kumar, N. and Coros, S., 2020. Soft Robot Control with A Learned Differentiable Model. International Conference on Soft Robotics, pp. 417-423.

Soft robots offer notable advantages over rigid robots in terms of flexibility and compliance, which facilitate safe and robust interactions with environments. These characteristics make them ideal for diverse applications, such as medical contexts and bionic robotics. However, due to the absence of precise models and the time-varying nature of their system dynamics, achieving optimal control performance on soft robots remains a significant challenge. With the rapid development of machine learning, we now have various learning-based methods to achieve precise pose control of soft robots [4]. Nevertheless, in practice, we have found that the (off-line) well-trained control policies often become ineffective upon deployment due to time-varying system dynamics, which lead us to rethink the traditional “sampling-training-deployment” paradigm in machine learning.

The rise of online learning can effectively enable models to adapt to changing system dynamics. Recent studies have shown that in online learning, even when using an approximate (or even linear) model, it is possible to ensure a sublinear convergence of the regret [2]. Additionally, online learning demonstrates high learning efficiency and stability in both simulations and complex real-world systems.

In this project, we will adopt gradient-based stochastic online learning to achieve precise pose control of a soft manipulator [1,3]. The cable-driven soft robot arm comprises three independent modules (about 0.2m for each module), and each module is actuated independently with three cables. More details of the robot setup, such as structural and mechanical description, can be found in our previous work [3]. We will first establish an approximate model of the system, then learn both feedforward and feedback controllers in an online manner, which enable the soft arm to perform a series of acrobatic demonstrations. Finally, we will compare our approach with some existing offline control algorithms to evaluate the performance in terms of learning efficiency, handling distribution shifts, and tracking accuracy.

Reference

[1] Chen, Z., Guan, Q., Hughes, J., Menciassi, A. and Stefanini, C., 2024. S2C2A: A Flexible Task Space Planning and Control Strategy for Modular Soft Robot Arms. arXiv preprint arXiv:2410.03483.

[2] Ma, H., Zeilinger, M. and Muehlebach, M., 2024. Stochastic Online Optimization for Cyber-Physical and Robotic Systems. arXiv preprint arXiv:2404.05318.

[3] Guan, Q., Stella, F., Della Santina, C., Leng, J. and Hughes, J., 2023. Trimmed Helicoids: An Architectured Soft Structure yielding Soft Robots with High Precision, Large Workspace, and Compliant Interactions. npj Robotics, 1(1), p. 4.

[4] Bern, J.M., Schnider, Y., Banzet, P., Kumar, N. and Coros, S., 2020. Soft Robot Control with A Learned Differentiable Model. International Conference on Soft Robotics, pp. 417-423.

Work packages Stage 1 - Preparation - Literature review on state-of-the-art work learning-based methods for soft robotics - Identify a rough (differentiable) model for the soft manipulator, e.g. system identification in the frequency domain, data-driven method - Setup the motion capture system for tracking the posture of the soft manipulator Stage 2 - Combine online learning & soft arm - Implement iterative learning control (ILC) for the soft manipulator to track a single fixed given reference trajectory - Implement stochastic online learning method for tracking any reference trajectories (in the workspace of the soft manipulator) Stage 3 - Analysis & demonstrations - Quantify the control performance, and compare to a baseline (e.g. an offline trained controller) - Perform some acrobatics using the soft manipulator for demonstration

Work packages

Stage 1 - Preparation

- Literature review on state-of-the-art work learning-based methods for soft robotics
- Identify a rough (differentiable) model for the soft manipulator, e.g. system identification in the frequency domain, data-driven method
- Setup the motion capture system for tracking the posture of the soft manipulator

Stage 2 - Combine online learning & soft arm

- Implement iterative learning control (ILC) for the soft manipulator to track a single fixed given reference trajectory
- Implement stochastic online learning method for tracking any reference trajectories (in the workspace of the soft manipulator)

Stage 3 - Analysis & demonstrations

- Quantify the control performance, and compare to a baseline (e.g. an offline trained controller)
- Perform some acrobatics using the soft manipulator for demonstration

Qinghua Guan, qinghua.guan@epfl.ch, EPFL Hao Ma, haomah@ethz.ch, ETHZ Cheng Pan, cheng.pan@epfl.ch, EPFL To apply, please include a short motivation for this project, as well as a copy of your CV and transcripts.

Qinghua Guan, qinghua.guan@epfl.ch, EPFL

Hao Ma, haomah@ethz.ch, ETHZ

Cheng Pan, cheng.pan@epfl.ch, EPFL

To apply, please include a short motivation for this project, as well as a copy of your CV and transcripts.