Multi-culture Sign Language Detection and Recognition Using Fine-tuned Convolutional Neural Network

M. Faisal Nurnoby, El Sayed M. El-Alfy

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Communication through hand gestures is an active research topic in computer vision and Human Computer Interaction (HCI). Deaf and speech-impaired community may benefit from the advancement of sign gesture recognition as an intermediary visual communication technique. There is no universal sign language, but different nationalities have their own sign languages. Usually sign gestures ought to be distinct and independently distinguishable and researchers in this domain have always focused on techniques for a particular sign language. However, recognizing sign languages from multiple nations could be helpful in various situations such as dealing with multiple signing persons without knowing their nationalities. This paper presents a novel approach where a two-phased system is designed based on deep learning for identification of the sign language and recognition of the sign gesture. This approach opens a new research direction for the advancement of multi-sign language development. The proposed approach has been implemented and evaluated using two custom-based CNN models and compared to some other popular CNN models. More than 98% accuracy can be achieved when tested on two different datasets.

Original languageEnglish
Title of host publicationInternational Conference on Smart Computing and Application, ICSCA 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798350347050
DOIs
StatePublished - 2023
Event2023 International Conference on Smart Computing and Application, ICSCA 2023 - Hail, Saudi Arabia
Duration: 5 Feb 20236 Feb 2023

Publication series

NameInternational Conference on Smart Computing and Application, ICSCA 2023

Conference

Conference2023 International Conference on Smart Computing and Application, ICSCA 2023
Country/TerritorySaudi Arabia
CityHail
Period5/02/236/02/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

  • Convolutional Neural Networks
  • Deep Learning
  • Gesture Recognition
  • Multi-culture Sign Language
  • Sign Language Identification

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Information Systems
  • Information Systems and Management

Fingerprint

Dive into the research topics of 'Multi-culture Sign Language Detection and Recognition Using Fine-tuned Convolutional Neural Network'. Together they form a unique fingerprint.

Cite this