Fuzziness Based Semi-supervised Deep Learning for Multimodal Image Classification

Abeda Asma; Dilshad Noor Mostafa; Koli Akter; Mufti Mahmud; Muhammed J.A. Patwary

doi:10.1007/978-3-031-34622-4_8

Fuzziness Based Semi-supervised Deep Learning for Multimodal Image Classification

Abeda Asma, Dilshad Noor Mostafa, Koli Akter, Mufti Mahmud, Muhammed J.A. Patwary^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

8 Scopus citations

Abstract

Predicting a class or label of text-aided image has practical application in a range of domains including social media, machine learning and medical domain. Usually, supervised learning model is used to make such predictions where labeled data is mandatory, which is time consuming and required manual help. Classification of images are accomplished on visual features only by utilizing deep learning. Employing semi-supervised learning is a viable answer to these issues that needs a few label sample to classify huge unlabeled samples. The paper suggests a novel semi-supervised deep learning method based on fuzziness, called (FSSDL-MIC) for multimodal image classification to tackle the challenge of web image classification. For the first time in this scenario, we integrate Multilayer perceptron for textual features and MobileNetV2 for visual features to create a multimodal paradigm. Using data from PASCAL VOC’07, experiments have revealed that the proposed framework achieves significant improvement and outperforms modern techniques for multimodal image categorization. We also see a positive impact of low fuzzy sample when final model trained with visual features only.

Original language	English
Title of host publication	Machine Intelligence and Emerging Technologies - First International Conference, MIET 2022, Proceedings
Editors	Md. Shahriare Satu, Mohammad Ali Moni, M. Shamim Kaiser, Mohammad Shamsul Arefin, Mohammad Shamsul Arefin
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	91-105
Number of pages	15
ISBN (Print)	9783031346217
DOIs	https://doi.org/10.1007/978-3-031-34622-4_8
State	Published - 2023
Externally published	Yes
Event	1st International Conference on Machine Intelligence and Emerging Technologies, MIET 2022 - Noakhali, Bangladesh Duration: 23 Sep 2022 → 25 Sep 2022

Publication series

Name	Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST
Volume	491 LNICST
ISSN (Print)	1867-8211
ISSN (Electronic)	1867-822X

Conference

Conference	1st International Conference on Machine Intelligence and Emerging Technologies, MIET 2022
Country/Territory	Bangladesh
City	Noakhali
Period	23/09/22 → 25/09/22

Bibliographical note

Publisher Copyright:
© 2023, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.

Keywords

Deep Learning
Fuzziness
Multimodal learning
Semi-supervised Learning

ASJC Scopus subject areas

Computer Networks and Communications

Access to Document

10.1007/978-3-031-34622-4_8

Cite this

Asma, A., Mostafa, D. N., Akter, K., Mahmud, M., & Patwary, M. J. A. (2023). Fuzziness Based Semi-supervised Deep Learning for Multimodal Image Classification. In M. S. Satu, M. A. Moni, M. S. Kaiser, M. S. Arefin, & M. S. Arefin (Eds.), Machine Intelligence and Emerging Technologies - First International Conference, MIET 2022, Proceedings (pp. 91-105). (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST; Vol. 491 LNICST). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-34622-4_8

Asma, Abeda ; Mostafa, Dilshad Noor ; Akter, Koli et al. / Fuzziness Based Semi-supervised Deep Learning for Multimodal Image Classification. Machine Intelligence and Emerging Technologies - First International Conference, MIET 2022, Proceedings. editor / Md. Shahriare Satu ; Mohammad Ali Moni ; M. Shamim Kaiser ; Mohammad Shamsul Arefin ; Mohammad Shamsul Arefin. Springer Science and Business Media Deutschland GmbH, 2023. pp. 91-105 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST).

@inproceedings{7ce3d6c41cbe49a188bba67b354b326a,

title = "Fuzziness Based Semi-supervised Deep Learning for Multimodal Image Classification",

abstract = "Predicting a class or label of text-aided image has practical application in a range of domains including social media, machine learning and medical domain. Usually, supervised learning model is used to make such predictions where labeled data is mandatory, which is time consuming and required manual help. Classification of images are accomplished on visual features only by utilizing deep learning. Employing semi-supervised learning is a viable answer to these issues that needs a few label sample to classify huge unlabeled samples. The paper suggests a novel semi-supervised deep learning method based on fuzziness, called (FSSDL-MIC) for multimodal image classification to tackle the challenge of web image classification. For the first time in this scenario, we integrate Multilayer perceptron for textual features and MobileNetV2 for visual features to create a multimodal paradigm. Using data from PASCAL VOC{\textquoteright}07, experiments have revealed that the proposed framework achieves significant improvement and outperforms modern techniques for multimodal image categorization. We also see a positive impact of low fuzzy sample when final model trained with visual features only.",

keywords = "Deep Learning, Fuzziness, Multimodal learning, Semi-supervised Learning",

author = "Abeda Asma and Mostafa, \{Dilshad Noor\} and Koli Akter and Mufti Mahmud and Patwary, \{Muhammed J.A.\}",

note = "Publisher Copyright: {\textcopyright} 2023, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.; 1st International Conference on Machine Intelligence and Emerging Technologies, MIET 2022 ; Conference date: 23-09-2022 Through 25-09-2022",

year = "2023",

doi = "10.1007/978-3-031-34622-4\_8",

language = "English",

isbn = "9783031346217",

series = "Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "91--105",

editor = "Satu, \{Md. Shahriare\} and Moni, \{Mohammad Ali\} and Kaiser, \{M. Shamim\} and Arefin, \{Mohammad Shamsul\} and Arefin, \{Mohammad Shamsul\}",

booktitle = "Machine Intelligence and Emerging Technologies - First International Conference, MIET 2022, Proceedings",

address = "Germany",

}

Asma, A, Mostafa, DN, Akter, K, Mahmud, M & Patwary, MJA 2023, Fuzziness Based Semi-supervised Deep Learning for Multimodal Image Classification. in MS Satu, MA Moni, MS Kaiser, MS Arefin & MS Arefin (eds), Machine Intelligence and Emerging Technologies - First International Conference, MIET 2022, Proceedings. Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, vol. 491 LNICST, Springer Science and Business Media Deutschland GmbH, pp. 91-105, 1st International Conference on Machine Intelligence and Emerging Technologies, MIET 2022, Noakhali, Bangladesh, 23/09/22. https://doi.org/10.1007/978-3-031-34622-4_8

Fuzziness Based Semi-supervised Deep Learning for Multimodal Image Classification. / Asma, Abeda; Mostafa, Dilshad Noor; Akter, Koli et al.
Machine Intelligence and Emerging Technologies - First International Conference, MIET 2022, Proceedings. ed. / Md. Shahriare Satu; Mohammad Ali Moni; M. Shamim Kaiser; Mohammad Shamsul Arefin; Mohammad Shamsul Arefin. Springer Science and Business Media Deutschland GmbH, 2023. p. 91-105 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST; Vol. 491 LNICST).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Fuzziness Based Semi-supervised Deep Learning for Multimodal Image Classification

AU - Asma, Abeda

AU - Mostafa, Dilshad Noor

AU - Akter, Koli

AU - Mahmud, Mufti

AU - Patwary, Muhammed J.A.

PY - 2023

Y1 - 2023

N2 - Predicting a class or label of text-aided image has practical application in a range of domains including social media, machine learning and medical domain. Usually, supervised learning model is used to make such predictions where labeled data is mandatory, which is time consuming and required manual help. Classification of images are accomplished on visual features only by utilizing deep learning. Employing semi-supervised learning is a viable answer to these issues that needs a few label sample to classify huge unlabeled samples. The paper suggests a novel semi-supervised deep learning method based on fuzziness, called (FSSDL-MIC) for multimodal image classification to tackle the challenge of web image classification. For the first time in this scenario, we integrate Multilayer perceptron for textual features and MobileNetV2 for visual features to create a multimodal paradigm. Using data from PASCAL VOC’07, experiments have revealed that the proposed framework achieves significant improvement and outperforms modern techniques for multimodal image categorization. We also see a positive impact of low fuzzy sample when final model trained with visual features only.

AB - Predicting a class or label of text-aided image has practical application in a range of domains including social media, machine learning and medical domain. Usually, supervised learning model is used to make such predictions where labeled data is mandatory, which is time consuming and required manual help. Classification of images are accomplished on visual features only by utilizing deep learning. Employing semi-supervised learning is a viable answer to these issues that needs a few label sample to classify huge unlabeled samples. The paper suggests a novel semi-supervised deep learning method based on fuzziness, called (FSSDL-MIC) for multimodal image classification to tackle the challenge of web image classification. For the first time in this scenario, we integrate Multilayer perceptron for textual features and MobileNetV2 for visual features to create a multimodal paradigm. Using data from PASCAL VOC’07, experiments have revealed that the proposed framework achieves significant improvement and outperforms modern techniques for multimodal image categorization. We also see a positive impact of low fuzzy sample when final model trained with visual features only.

KW - Deep Learning

KW - Fuzziness

KW - Multimodal learning

KW - Semi-supervised Learning

UR - http://www.scopus.com/inward/record.url?scp=85164102194&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-34622-4_8

DO - 10.1007/978-3-031-34622-4_8

M3 - Conference contribution

AN - SCOPUS:85164102194

SN - 9783031346217

T3 - Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST

SP - 91

EP - 105

BT - Machine Intelligence and Emerging Technologies - First International Conference, MIET 2022, Proceedings

A2 - Satu, Md. Shahriare

A2 - Moni, Mohammad Ali

A2 - Kaiser, M. Shamim

A2 - Arefin, Mohammad Shamsul

PB - Springer Science and Business Media Deutschland GmbH

T2 - 1st International Conference on Machine Intelligence and Emerging Technologies, MIET 2022

Y2 - 23 September 2022 through 25 September 2022

ER -

Asma A, Mostafa DN, Akter K, Mahmud M, Patwary MJA. Fuzziness Based Semi-supervised Deep Learning for Multimodal Image Classification. In Satu MS, Moni MA, Kaiser MS, Arefin MS, Arefin MS, editors, Machine Intelligence and Emerging Technologies - First International Conference, MIET 2022, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 91-105. (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST). doi: 10.1007/978-3-031-34622-4_8