S-Space College of Engineering/Engineering Practice School (공과대학/대학원) Dept. of Mechanical Aerospace Engineering (기계항공공학부) Theses (Master's Degree_기계항공공학부)
Learning and Generalization of Dynamic Movement Primitives by Hierarchical Deep Reinforcement Learning
계층적 심층 강화학습을 활용한 동적 단위 동작의 학습 및 일반화
- 공과대학 기계항공공학부
- Issue Date
- 서울대학교 대학원
- 학위논문 (석사)-- 서울대학교 대학원 : 공과대학 기계항공공학부, 2018. 8. 김현진.
- This paper presents an approach to learn and generalize robotic skills from a demonstration using deep
reinforcement learning (deep RL). Dynamic Movement Primitives (DMPs) formulate a nonlinear differential
equation and produce the observed movement from a demonstration. However, it is hard to generate
new behaviors from using DMPs. Thus, we apply DMPs framework into deep RL as an initial setting for
learning the robotic skills. First, we build a network to represent this differential equation, and learn and
generalize the movements by optimizing the shape of DMPs with respect to the rewards up to the end of
each sequence of movement primitives. In order to do this, we consider a deterministic actor-critic algorithm
for deep RL and we also apply a hierarchical strategy. This drastically reduces the search space for
a robot by decomposing the task, which allows to solve the sparse reward problem from a complex task.
In order to integrate DMPs with hierarchical deep RL, the differential equation is considered as temporal
abstraction of option. The overall structure is mainly composed of two controllers: meta-controller and
sub-controller. The meta-controller learns a policy over intrinsic goals and a sub-controller learns a policy
over actions to accomplish the given goals. We demonstrate our approach on a 6 degree-of-freedom
(DOF) arm with a 1-DOF gripper and evaluate that DMPs are learned and generalized using deep RL with
a pick-and-place task.