Transformers for Low-Resource Religious Hate Speech Detection: A Comparative Study on Bangla Social Media Discourse

Religious hate speech detection in the web for low-resource languages like Bangla presents significant challenges due to linguistic complexity and socio-cultural nuances. This study evaluates the effectiveness of six transformer-based models on a Bangla religious hate speech dataset sourced from social media comments targeting Hindu and Muslim communities in Bangladesh and India. Among the models tested, DistilBERT, XLNet, and ELECTRA demonstrated superior performance. DistilBERT achieved the highest overall accuracy of approximately 91.7%, along with a strong F1 score close to 92%, supported by an impressive recall of over 95% and a ROC AUC of 0.98. XLNet showed outstanding precision at 98% and a ROC AUC of 0.979, while ELECTRA balanced precision and recall effectively, delivering around 91.5% accuracy and a comparable F1 score. These results highlight the potential of transformer architectures to advance hate speech detection in low-resource languages such as Bangla, offering promising foundations for automated content moderation tools that can help mitigate online religious intolerance. Future work will focus on improving model robustness through domain-adaptive pretraining, exploring cross-lingual transfer learning to extend capabilities to other South Asian low-resource languages and developing scalable real-time moderation frameworks for practical deployment in social media platforms.