Using Transformers to Classify Arabic Dialects on Social Networks

Arabic dialects are the history and civilization of Arab societies and distinguish people and cities from each other in cultures, traditions, and customs. Social media networks provide speakers of various dialects with spaces through comments and tweets to spread their culture and traditions in local and regional dialects. In this research, we propose an intelligent method that extracts attention-grabbing words in the text and extracts their features and connections after converting them into digital data for processing. This research classifies these texts using the latest natural language processing applications, namely transformers, based on the attention mechanism. We implemented two transformer models: BERT with Deep Learning (Bidirectional Encoded Representations of Transformers) and DistillBERT with Machine Learning, a small, fast, cheap, and lightweight transformer model trained by the BERT Distilling Base. Both are pre-trained text encoders, and we implemented the new regularization approach for the final classification task. Our experiment used a text dataset for five Arabic dialects (Algerian, Tunisian, Egyptian, Levantine, and Gulf Arabic). Both models provide outstanding performance on the dataset regarding accuracy and speed.