Intelligent Biped Robot Simulation Locomotion using Deep Q Network

Hasan Imaduddin; Azhar Aulia Saputra; Indra Adji Sulistijono; Naoyuki Kubota

doi:10.1109/IES55876.2022.9888748

Intelligent Biped Robot Simulation Locomotion using Deep Q Network

Hasan Imaduddin
, Azhar Aulia Saputra
, Indra Adji Sulistijono
, Naoyuki Kubota

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

This research suggests a deep Q-network learning approach coupled with a layered neural-based locomotion. An analog signal is produced by central pattern generation (CPG) in the higher layers and sent to motor neuron pools in the lower levels. Motor neurons produce joint angular velocity in the lower levels. The neural oscillator model in the spinal cord serves as the foundation for the central pattern generation. It reacts to the pattern of the driving model. Then, using a human musculoskeletal model, the inner state of motor neurons was constructed based on muscle activation. Additionally, the motor neuron (MN) pool is linked to the sensory neuron (SN), which transmits internal feedback from the robot. A humanoid robot simulation using the proposed concept has been used to carry various loads. Adjustments to the robot's behavior to maintain its stance.

Original language	English
Title of host publication	IES 2022 - 2022 International Electronics Symposium
Subtitle of host publication	Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding
Editors	Andhik Ampuh Yunanto, Yanuar Risah Prayogi, Putu Agus Mahadi Putra, Hendhi Hermawan, Nailussa'ada Nailussa'ada, Maretha Ruswiansari, Mohamad Ridwan, Farida Gamar, Afifah Dwi Ramadhani, Weny Mistarika Rahmawati, Muhammad Rizani Rusli
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	324-329
Number of pages	6
ISBN (Electronic)	9781665489713
DOIs	https://doi.org/10.1109/IES55876.2022.9888748
State	Published - 2022
Externally published	Yes
Event	24th International Electronics Symposium, IES 2022 - Surabaya, Indonesia Duration: 9 Aug 2022 → 11 Aug 2022

Publication series

Name	IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding

Conference

Conference	24th International Electronics Symposium, IES 2022
Country/Territory	Indonesia
City	Surabaya
Period	9/08/22 → 11/08/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Keywords

central pattern generation
deep q network
neural-based locomotion

ASJC Scopus subject areas

Energy Engineering and Power Technology
Renewable Energy, Sustainability and the Environment
Electrical and Electronic Engineering
Global and Planetary Change
Control and Optimization

Access to Document

10.1109/IES55876.2022.9888748

Cite this

Imaduddin, H., Saputra, A. A., Sulistijono, I. A., & Kubota, N. (2022). Intelligent Biped Robot Simulation Locomotion using Deep Q Network. In A. A. Yunanto, Y. R. Prayogi, P. A. M. Putra, H. Hermawan, N. Nailussa'ada, M. Ruswiansari, M. Ridwan, F. Gamar, A. D. Ramadhani, W. M. Rahmawati, & M. R. Rusli (Eds.), IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding (pp. 324-329). (IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IES55876.2022.9888748

Imaduddin, Hasan ; Saputra, Azhar Aulia ; Sulistijono, Indra Adji et al. / Intelligent Biped Robot Simulation Locomotion using Deep Q Network. IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding. editor / Andhik Ampuh Yunanto ; Yanuar Risah Prayogi ; Putu Agus Mahadi Putra ; Hendhi Hermawan ; Nailussa'ada Nailussa'ada ; Maretha Ruswiansari ; Mohamad Ridwan ; Farida Gamar ; Afifah Dwi Ramadhani ; Weny Mistarika Rahmawati ; Muhammad Rizani Rusli. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 324-329 (IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding).

@inproceedings{206bb611f8b943229a42287c42fff8db,

title = "Intelligent Biped Robot Simulation Locomotion using Deep Q Network",

abstract = "This research suggests a deep Q-network learning approach coupled with a layered neural-based locomotion. An analog signal is produced by central pattern generation (CPG) in the higher layers and sent to motor neuron pools in the lower levels. Motor neurons produce joint angular velocity in the lower levels. The neural oscillator model in the spinal cord serves as the foundation for the central pattern generation. It reacts to the pattern of the driving model. Then, using a human musculoskeletal model, the inner state of motor neurons was constructed based on muscle activation. Additionally, the motor neuron (MN) pool is linked to the sensory neuron (SN), which transmits internal feedback from the robot. A humanoid robot simulation using the proposed concept has been used to carry various loads. Adjustments to the robot's behavior to maintain its stance.",

keywords = "central pattern generation, deep q network, neural-based locomotion",

author = "Hasan Imaduddin and Saputra, \{Azhar Aulia\} and Sulistijono, \{Indra Adji\} and Naoyuki Kubota",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 24th International Electronics Symposium, IES 2022 ; Conference date: 09-08-2022 Through 11-08-2022",

year = "2022",

doi = "10.1109/IES55876.2022.9888748",

language = "English",

series = "IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "324--329",

editor = "Yunanto, \{Andhik Ampuh\} and Prayogi, \{Yanuar Risah\} and Putra, \{Putu Agus Mahadi\} and Hendhi Hermawan and Nailussa'ada Nailussa'ada and Maretha Ruswiansari and Mohamad Ridwan and Farida Gamar and Ramadhani, \{Afifah Dwi\} and Rahmawati, \{Weny Mistarika\} and Rusli, \{Muhammad Rizani\}",

booktitle = "IES 2022 - 2022 International Electronics Symposium",

address = "United States",

}

Imaduddin, H, Saputra, AA, Sulistijono, IA & Kubota, N 2022, Intelligent Biped Robot Simulation Locomotion using Deep Q Network. in AA Yunanto, YR Prayogi, PAM Putra, H Hermawan, N Nailussa'ada, M Ruswiansari, M Ridwan, F Gamar, AD Ramadhani, WM Rahmawati & MR Rusli (eds), IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding. IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding, Institute of Electrical and Electronics Engineers Inc., pp. 324-329, 24th International Electronics Symposium, IES 2022, Surabaya, Indonesia, 9/08/22. https://doi.org/10.1109/IES55876.2022.9888748

Intelligent Biped Robot Simulation Locomotion using Deep Q Network. / Imaduddin, Hasan; Saputra, Azhar Aulia; Sulistijono, Indra Adji et al.
IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding. ed. / Andhik Ampuh Yunanto; Yanuar Risah Prayogi; Putu Agus Mahadi Putra; Hendhi Hermawan; Nailussa'ada Nailussa'ada; Maretha Ruswiansari; Mohamad Ridwan; Farida Gamar; Afifah Dwi Ramadhani; Weny Mistarika Rahmawati; Muhammad Rizani Rusli. Institute of Electrical and Electronics Engineers Inc., 2022. p. 324-329 (IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Intelligent Biped Robot Simulation Locomotion using Deep Q Network

AU - Imaduddin, Hasan

AU - Saputra, Azhar Aulia

AU - Sulistijono, Indra Adji

AU - Kubota, Naoyuki

PY - 2022

Y1 - 2022

N2 - This research suggests a deep Q-network learning approach coupled with a layered neural-based locomotion. An analog signal is produced by central pattern generation (CPG) in the higher layers and sent to motor neuron pools in the lower levels. Motor neurons produce joint angular velocity in the lower levels. The neural oscillator model in the spinal cord serves as the foundation for the central pattern generation. It reacts to the pattern of the driving model. Then, using a human musculoskeletal model, the inner state of motor neurons was constructed based on muscle activation. Additionally, the motor neuron (MN) pool is linked to the sensory neuron (SN), which transmits internal feedback from the robot. A humanoid robot simulation using the proposed concept has been used to carry various loads. Adjustments to the robot's behavior to maintain its stance.

AB - This research suggests a deep Q-network learning approach coupled with a layered neural-based locomotion. An analog signal is produced by central pattern generation (CPG) in the higher layers and sent to motor neuron pools in the lower levels. Motor neurons produce joint angular velocity in the lower levels. The neural oscillator model in the spinal cord serves as the foundation for the central pattern generation. It reacts to the pattern of the driving model. Then, using a human musculoskeletal model, the inner state of motor neurons was constructed based on muscle activation. Additionally, the motor neuron (MN) pool is linked to the sensory neuron (SN), which transmits internal feedback from the robot. A humanoid robot simulation using the proposed concept has been used to carry various loads. Adjustments to the robot's behavior to maintain its stance.

KW - central pattern generation

KW - deep q network

KW - neural-based locomotion

UR - https://www.scopus.com/pages/publications/85139658623

U2 - 10.1109/IES55876.2022.9888748

DO - 10.1109/IES55876.2022.9888748

M3 - Conference contribution

AN - SCOPUS:85139658623

T3 - IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding

SP - 324

EP - 329

BT - IES 2022 - 2022 International Electronics Symposium

A2 - Yunanto, Andhik Ampuh

A2 - Prayogi, Yanuar Risah

A2 - Putra, Putu Agus Mahadi

A2 - Hermawan, Hendhi

A2 - Nailussa'ada, Nailussa'ada

A2 - Ruswiansari, Maretha

A2 - Ridwan, Mohamad

A2 - Gamar, Farida

A2 - Ramadhani, Afifah Dwi

A2 - Rahmawati, Weny Mistarika

A2 - Rusli, Muhammad Rizani

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 24th International Electronics Symposium, IES 2022

Y2 - 9 August 2022 through 11 August 2022

ER -

Imaduddin H, Saputra AA, Sulistijono IA, Kubota N. Intelligent Biped Robot Simulation Locomotion using Deep Q Network. In Yunanto AA, Prayogi YR, Putra PAM, Hermawan H, Nailussa'ada N, Ruswiansari M, Ridwan M, Gamar F, Ramadhani AD, Rahmawati WM, Rusli MR, editors, IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding. Institute of Electrical and Electronics Engineers Inc. 2022. p. 324-329. (IES 2022 - 2022 International Electronics Symposium: Energy Development for Climate Change Solution and Clean Energy Transition, Proceeding). doi: 10.1109/IES55876.2022.9888748

Intelligent Biped Robot Simulation Locomotion using Deep Q Network

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this