Robot intelligent trajectory planning based on PCM guided reinforcement learning

Xiang Teng; Jian Fu; Cong Li; Zhaojie Ju

doi:10.1007/978-3-030-27529-7_30

Robot intelligent trajectory planning based on PCM guided reinforcement learning

Xiang Teng, Jian Fu, Cong Li, Zhaojie Ju

School of Computing

Wuhan University of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

171 Downloads (Pure)

Abstract

Reinforcement Learning (RL) was successfully applied in multi-degree-of-freedoms robot to acquire motor skills, however, it hardly ever consider each joints’ relationship, or just think about the linear relationship between them. In order to find the nonlinear relationship between each degrees of freedom (DOFs), we propose a Pseudo Covariance Matrix (PCM) to guide reinforcement learning for motor skill acquisition. Specifically it combined Path Integral Policy Improvement (PI2) with Kernel Canonical Correlation Analysis (KCCA), where KCCA is used to obtain the PCM in high dimensional space and record it as the heuristic information to search an optimal/sub-optimal strategy. The experiments based on robots (SCARA and UR5) demonstrate the new method is feasible and effective.

Original language	English
Title of host publication	Intelligent Robotics and Applications
Subtitle of host publication	12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part VI
Editors	Haibin Yu, Jinguo Liu, Lianqing Liu, Zhaojie Ju, Yuwang Liu, Dalin Zhou
Publisher	Springer
Chapter	30
Pages	342-355
Number of pages	14
ISBN (Electronic)	978-3-030-27529-7
ISBN (Print)	978-3-030-27528-0
DOIs	https://doi.org/10.1007/978-3-030-27529-7_30
Publication status	Published - 6 Aug 2019
Event	12th International Conference on Intelligent Robotics and Applications - Shenyang, China Duration: 8 Aug 2019 → 11 Aug 2019

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	11745
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Name	Lecture Notes in Artificial Intelligence
Publisher	Springer
Volume	11745
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	12th International Conference on Intelligent Robotics and Applications
Abbreviated title	ICIRA 2019
Country/Territory	China
City	Shenyang
Period	8/08/19 → 11/08/19

Keywords

trajectory planning
learning from demonstration
Kernel canonical correlation analysis
path integral policy improvement
pseudo covariance matrix

Access to Document

10.1007/978-3-030-27529-7_30

Teng_Robot Intelligent Trajectory_ICIRA2019_233_final_v5-2
This is a post-peer-review, pre-copyedit version of an article published in Intelligent Robotics and Applications. ICIRA 2019. The final authenticated version is available online at: http://dx.doi.org/10.1007/978-3-030-27529-7_30.
Accepted author manuscript (Post-print), 2.55 MB

Cite this

Teng, X., Fu, J., Li, C., & Ju, Z. (2019). Robot intelligent trajectory planning based on PCM guided reinforcement learning. In H. Yu, J. Liu, L. Liu, Z. Ju, Y. Liu, & D. Zhou (Eds.), Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part VI (pp. 342-355). (Lecture Notes in Computer Science; Vol. 11745), (Lecture Notes in Artificial Intelligence; Vol. 11745). Springer. https://doi.org/10.1007/978-3-030-27529-7_30

Teng, Xiang ; Fu, Jian ; Li, Cong et al. / Robot intelligent trajectory planning based on PCM guided reinforcement learning. Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part VI. editor / Haibin Yu ; Jinguo Liu ; Lianqing Liu ; Zhaojie Ju ; Yuwang Liu ; Dalin Zhou. Springer, 2019. pp. 342-355 (Lecture Notes in Computer Science). (Lecture Notes in Artificial Intelligence).

@inproceedings{5ee0148368f54a33b3220fbad13e7b34,

title = "Robot intelligent trajectory planning based on PCM guided reinforcement learning",

abstract = "Reinforcement Learning (RL) was successfully applied in multi-degree-of-freedoms robot to acquire motor skills, however, it hardly ever consider each joints{\textquoteright} relationship, or just think about the linear relationship between them. In order to find the nonlinear relationship between each degrees of freedom (DOFs), we propose a Pseudo Covariance Matrix (PCM) to guide reinforcement learning for motor skill acquisition. Specifically it combined Path Integral Policy Improvement (PI2) with Kernel Canonical Correlation Analysis (KCCA), where KCCA is used to obtain the PCM in high dimensional space and record it as the heuristic information to search an optimal/sub-optimal strategy. The experiments based on robots (SCARA and UR5) demonstrate the new method is feasible and effective.",

keywords = "trajectory planning, learning from demonstration, Kernel canonical correlation analysis, path integral policy improvement, pseudo covariance matrix",

author = "Xiang Teng and Jian Fu and Cong Li and Zhaojie Ju",

year = "2019",

month = aug,

day = "6",

doi = "10.1007/978-3-030-27529-7_30",

language = "English",

isbn = "978-3-030-27528-0",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "342--355",

editor = "Haibin Yu and Jinguo Liu and Lianqing Liu and Zhaojie Ju and Yuwang Liu and Dalin Zhou",

booktitle = "Intelligent Robotics and Applications",

note = "12th International Conference on Intelligent Robotics and Applications, ICIRA 2019 ; Conference date: 08-08-2019 Through 11-08-2019",

}

Teng, X, Fu, J, Li, C & Ju, Z 2019, Robot intelligent trajectory planning based on PCM guided reinforcement learning. in H Yu, J Liu, L Liu, Z Ju, Y Liu & D Zhou (eds), Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part VI. Lecture Notes in Computer Science, vol. 11745, Lecture Notes in Artificial Intelligence, vol. 11745, Springer, pp. 342-355, 12th International Conference on Intelligent Robotics and Applications, Shenyang, China, 8/08/19. https://doi.org/10.1007/978-3-030-27529-7_30

Robot intelligent trajectory planning based on PCM guided reinforcement learning. / Teng, Xiang; Fu, Jian; Li, Cong et al.
Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part VI. ed. / Haibin Yu; Jinguo Liu; Lianqing Liu; Zhaojie Ju; Yuwang Liu; Dalin Zhou. Springer, 2019. p. 342-355 (Lecture Notes in Computer Science; Vol. 11745), (Lecture Notes in Artificial Intelligence; Vol. 11745).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Robot intelligent trajectory planning based on PCM guided reinforcement learning

AU - Teng, Xiang

AU - Fu, Jian

AU - Li, Cong

AU - Ju, Zhaojie

PY - 2019/8/6

Y1 - 2019/8/6

N2 - Reinforcement Learning (RL) was successfully applied in multi-degree-of-freedoms robot to acquire motor skills, however, it hardly ever consider each joints’ relationship, or just think about the linear relationship between them. In order to find the nonlinear relationship between each degrees of freedom (DOFs), we propose a Pseudo Covariance Matrix (PCM) to guide reinforcement learning for motor skill acquisition. Specifically it combined Path Integral Policy Improvement (PI2) with Kernel Canonical Correlation Analysis (KCCA), where KCCA is used to obtain the PCM in high dimensional space and record it as the heuristic information to search an optimal/sub-optimal strategy. The experiments based on robots (SCARA and UR5) demonstrate the new method is feasible and effective.

AB - Reinforcement Learning (RL) was successfully applied in multi-degree-of-freedoms robot to acquire motor skills, however, it hardly ever consider each joints’ relationship, or just think about the linear relationship between them. In order to find the nonlinear relationship between each degrees of freedom (DOFs), we propose a Pseudo Covariance Matrix (PCM) to guide reinforcement learning for motor skill acquisition. Specifically it combined Path Integral Policy Improvement (PI2) with Kernel Canonical Correlation Analysis (KCCA), where KCCA is used to obtain the PCM in high dimensional space and record it as the heuristic information to search an optimal/sub-optimal strategy. The experiments based on robots (SCARA and UR5) demonstrate the new method is feasible and effective.

KW - trajectory planning

KW - learning from demonstration

KW - Kernel canonical correlation analysis

KW - path integral policy improvement

KW - pseudo covariance matrix

U2 - 10.1007/978-3-030-27529-7_30

DO - 10.1007/978-3-030-27529-7_30

M3 - Conference contribution

SN - 978-3-030-27528-0

T3 - Lecture Notes in Computer Science

SP - 342

EP - 355

BT - Intelligent Robotics and Applications

A2 - Yu, Haibin

A2 - Liu, Jinguo

A2 - Liu, Lianqing

A2 - Ju, Zhaojie

A2 - Liu, Yuwang

A2 - Zhou, Dalin

PB - Springer

T2 - 12th International Conference on Intelligent Robotics and Applications

Y2 - 8 August 2019 through 11 August 2019

ER -

Teng X, Fu J, Li C, Ju Z. Robot intelligent trajectory planning based on PCM guided reinforcement learning. In Yu H, Liu J, Liu L, Ju Z, Liu Y, Zhou D, editors, Intelligent Robotics and Applications: 12th International Conference, ICIRA 2019, Shenyang, China, August 8–11, 2019, Proceedings, Part VI. Springer. 2019. p. 342-355. (Lecture Notes in Computer Science). (Lecture Notes in Artificial Intelligence). doi: 10.1007/978-3-030-27529-7_30

Robot intelligent trajectory planning based on PCM guided reinforcement learning

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this