Deep reinforcement learning with light-weight vision model for sequential robotic object sorting

Emmanuel Okafor*, Mojeed Oyedeji, Motaz Alfarraj

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Training an agent to generate cooperative joint learning policies for executing object sorting in cluttered scenes is complex. This paper presents a comparative model-free deep reinforcement learning (DRL) system built from variants of end-to-end lightweight deep neural networks that jointly complement three primitive policies (pushing, grasping, and placing) that operate from visual observations to actions. The end goal of the proposed DRL systems is to perform object sorting in very dynamic and complex environmental scenarios of cluttered and random objects (irregular and regular blocks) existing in different color variations. For this, we investigated DRL methods built from 12 custom instances of Pixel-wise Q-valued Critic Networks (PQCN) from four main backbone networks (DenseNet121, DenseNet169, MobileNetV3, and SqueezeNet) individually combined with custom fully convolutional neural networks (FCN) for learning the image-based observation space through affordance mapping paradigm while considering both dual and single transfer learning schemes. Additionally, we explored three categories of gradient-based optimization methods and considered custom reward functions with varying discount factors. The results show that the PQCN constructed by integrating DenseNet121 or MobileNetV3 with FCN and trained by a dual transfer learning scheme (with a discount factor of γ=0.70) yielded the best performance in most categories of our evaluation in the training phases. Overall, the PQCN-DenseNet121, when trained with a dual transfer learning process, yielded the best generalization (sorting success rate) in most of the evaluation metrics across all object categories in the testing phase. Furthermore, our study provides a new benchmark for validating deep reinforcement learning algorithms while sorting object blocks under varying degrees of complexity. This research finds practical relevance in industrial fields such as manufacturing and packaging, building construction, plant sorting, automobile assembly lines, and waste sorting. Here is the article's source code: https://github.com/Emmanuel-Okafor/DRL_for_Object_Sorting.git.

Original languageEnglish
Article number101896
JournalJournal of King Saud University - Computer and Information Sciences
Volume36
Issue number1
DOIs
StatePublished - Jan 2024

Bibliographical note

Publisher Copyright:
© 2023 The Authors

Keywords

  • Artificial intelligence
  • Object sorting
  • Reinforcement learning
  • Reward engineering

ASJC Scopus subject areas

  • General Computer Science

Fingerprint

Dive into the research topics of 'Deep reinforcement learning with light-weight vision model for sequential robotic object sorting'. Together they form a unique fingerprint.

Cite this