ArabSign: A Multi-modality Dataset and Benchmark for Continuous Arabic Sign Language Recognition

Hamzah Luqman*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

Sign language recognition has attracted the interest of researchers in recent years. While numerous approaches have been proposed for European and Asian sign languages recognition, very limited attempts have been made to develop similar systems for the Arabic sign language (ArSL). This can be attributed partly to the lack of a dataset at the sentence level. In this paper, we aim to make a significant contribution by proposing ArabSign, a continuous ArSL dataset. The proposed dataset consists of 9,335 samples performed by 6 signers. The total time of the recorded sentences is around 10 hours and the average sentence's length is 3.1 signs. ArabSign dataset was recorded using a Kinect V2 camera that provides three types of information (color, depth, and skeleton joint points) recorded simultaneously for each sentence. In addition, we provide the annotation of the dataset according to ArSL and Arabic language structures that can help in studying the linguistic characteristics of ArSL. To benchmark this dataset, we propose an encoder-decoder model for Continuous ArSL recognition. The model has been evaluated on the proposed dataset, and the obtained results show that the encoder-decoder model outperformed the attention mechanism with an average word error rate (WER) of 0.50 compared with 0.62 with the attention mechanism. The data and code are available at https://github.com/Hamzah-Luqman/rabSign

Original languageEnglish
Title of host publication2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition, FG 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798350345445
DOIs
StatePublished - 2023
Event17th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2023 - Waikoloa Beach, United States
Duration: 5 Jan 20238 Jan 2023

Publication series

Name2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition, FG 2023

Conference

Conference17th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2023
Country/TerritoryUnited States
CityWaikoloa Beach
Period5/01/238/01/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

ASJC Scopus subject areas

  • Computer Science Applications
  • Artificial Intelligence
  • Computer Vision and Pattern Recognition

Fingerprint

Dive into the research topics of 'ArabSign: A Multi-modality Dataset and Benchmark for Continuous Arabic Sign Language Recognition'. Together they form a unique fingerprint.

Cite this