Arabic Keyphrase Extraction: Enhancing Deep Learning Models with Pre-trained Contextual Embedding and External Features

Randah Alharbi; Husni Al-Muhtaseb

doi:10.18653/v1/2022.wanlp-1.30

Arabic Keyphrase Extraction: Enhancing Deep Learning Models with Pre-trained Contextual Embedding and External Features

Randah Alharbi, Husni Al-Muhtaseb

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Keyphrase extraction is essential to many Information retrieval (IR) and Natural language Processing (NLP) tasks such as summarization and indexing. This study investigates deep learning approaches to Arabic keyphrase extraction. We address the problem as sequence classification and create a Bi-LSTM model to classify each sequence token as either part of the keyphrase or outside of it. We have extracted word embeddings from two pre-trained models, Word2Vec and BERT. Moreover, we have investigated the effect of incorporating linguistic, positional, and statistical features with word embeddings on performance. Our best-performing model has achieved 0.45 F1-score on ArabicKPE dataset when combining linguistic and positional features with BERT embedding.

Original language	English
Title of host publication	WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop
Publisher	Association for Computational Linguistics (ACL)
Pages	320-330
Number of pages	11
ISBN (Electronic)	9781959429272
DOIs	https://doi.org/10.18653/v1/2022.wanlp-1.30
State	Published - 2022
Event	7th Arabic Natural Language Processing Workshop, WANLP 2022 held with EMNLP 2022 - Abu Dhabi, United Arab Emirates Duration: 8 Dec 2022 → …

Publication series

Name	WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop

Conference

Conference	7th Arabic Natural Language Processing Workshop, WANLP 2022 held with EMNLP 2022
Country/Territory	United Arab Emirates
City	Abu Dhabi
Period	8/12/22 → …

Bibliographical note

Publisher Copyright:
© 2022 Association for Computational Linguistics.

ASJC Scopus subject areas

Language and Linguistics
Computational Theory and Mathematics
Software
Linguistics and Language

Access to Document

10.18653/v1/2022.wanlp-1.30

Cite this

Alharbi, R., & Al-Muhtaseb, H. (2022). Arabic Keyphrase Extraction: Enhancing Deep Learning Models with Pre-trained Contextual Embedding and External Features. In WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop (pp. 320-330). (WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2022.wanlp-1.30

Alharbi, Randah ; Al-Muhtaseb, Husni. / Arabic Keyphrase Extraction : Enhancing Deep Learning Models with Pre-trained Contextual Embedding and External Features. WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop. Association for Computational Linguistics (ACL), 2022. pp. 320-330 (WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop).

@inproceedings{37a3d5e57ba34371a87425a9fdb22f84,

title = "Arabic Keyphrase Extraction: Enhancing Deep Learning Models with Pre-trained Contextual Embedding and External Features",

abstract = "Keyphrase extraction is essential to many Information retrieval (IR) and Natural language Processing (NLP) tasks such as summarization and indexing. This study investigates deep learning approaches to Arabic keyphrase extraction. We address the problem as sequence classification and create a Bi-LSTM model to classify each sequence token as either part of the keyphrase or outside of it. We have extracted word embeddings from two pre-trained models, Word2Vec and BERT. Moreover, we have investigated the effect of incorporating linguistic, positional, and statistical features with word embeddings on performance. Our best-performing model has achieved 0.45 F1-score on ArabicKPE dataset when combining linguistic and positional features with BERT embedding.",

author = "Randah Alharbi and Husni Al-Muhtaseb",

note = "Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.; 7th Arabic Natural Language Processing Workshop, WANLP 2022 held with EMNLP 2022 ; Conference date: 08-12-2022",

year = "2022",

doi = "10.18653/v1/2022.wanlp-1.30",

language = "English",

series = "WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop",

publisher = "Association for Computational Linguistics (ACL)",

pages = "320--330",

booktitle = "WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop",

address = "United States",

}

Alharbi, R & Al-Muhtaseb, H 2022, Arabic Keyphrase Extraction: Enhancing Deep Learning Models with Pre-trained Contextual Embedding and External Features. in WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop. WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop, Association for Computational Linguistics (ACL), pp. 320-330, 7th Arabic Natural Language Processing Workshop, WANLP 2022 held with EMNLP 2022, Abu Dhabi, United Arab Emirates, 8/12/22. https://doi.org/10.18653/v1/2022.wanlp-1.30

Arabic Keyphrase Extraction: Enhancing Deep Learning Models with Pre-trained Contextual Embedding and External Features. / Alharbi, Randah; Al-Muhtaseb, Husni.
WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop. Association for Computational Linguistics (ACL), 2022. p. 320-330 (WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Arabic Keyphrase Extraction

T2 - 7th Arabic Natural Language Processing Workshop, WANLP 2022 held with EMNLP 2022

AU - Alharbi, Randah

AU - Al-Muhtaseb, Husni

PY - 2022

Y1 - 2022

N2 - Keyphrase extraction is essential to many Information retrieval (IR) and Natural language Processing (NLP) tasks such as summarization and indexing. This study investigates deep learning approaches to Arabic keyphrase extraction. We address the problem as sequence classification and create a Bi-LSTM model to classify each sequence token as either part of the keyphrase or outside of it. We have extracted word embeddings from two pre-trained models, Word2Vec and BERT. Moreover, we have investigated the effect of incorporating linguistic, positional, and statistical features with word embeddings on performance. Our best-performing model has achieved 0.45 F1-score on ArabicKPE dataset when combining linguistic and positional features with BERT embedding.

AB - Keyphrase extraction is essential to many Information retrieval (IR) and Natural language Processing (NLP) tasks such as summarization and indexing. This study investigates deep learning approaches to Arabic keyphrase extraction. We address the problem as sequence classification and create a Bi-LSTM model to classify each sequence token as either part of the keyphrase or outside of it. We have extracted word embeddings from two pre-trained models, Word2Vec and BERT. Moreover, we have investigated the effect of incorporating linguistic, positional, and statistical features with word embeddings on performance. Our best-performing model has achieved 0.45 F1-score on ArabicKPE dataset when combining linguistic and positional features with BERT embedding.

UR - http://www.scopus.com/inward/record.url?scp=85152911797&partnerID=8YFLogxK

U2 - 10.18653/v1/2022.wanlp-1.30

DO - 10.18653/v1/2022.wanlp-1.30

M3 - Conference contribution

AN - SCOPUS:85152911797

T3 - WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop

SP - 320

EP - 330

BT - WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop

PB - Association for Computational Linguistics (ACL)

Y2 - 8 December 2022

ER -

Alharbi R, Al-Muhtaseb H. Arabic Keyphrase Extraction: Enhancing Deep Learning Models with Pre-trained Contextual Embedding and External Features. In WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop. Association for Computational Linguistics (ACL). 2022. p. 320-330. (WANLP 2022 - 7th Arabic Natural Language Processing - Proceedings of the Workshop). doi: 10.18653/v1/2022.wanlp-1.30

Arabic Keyphrase Extraction: Enhancing Deep Learning Models with Pre-trained Contextual Embedding and External Features

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this