ARiViT: attention-based residual-integrated vision transformer for noisy brain medical image classification

Brain tumor detection in medical image processing presents a formidable challenge due to the complex behavior exhibited by these tumors. Their intricate nature arises from a variety of shapes and textures within brain tumor images, exacerbated by their diverse cellular origins. This complexity is further compounded by the presence of noisy images, which add an additional layer of difficulty. The overlapping image intensities in tumor and non-tumor areas pose a significant hurdle in extracting meaningful insights from raw data using predictive models. In response to these challenges, we introduce ARiViT, a novel framework based on residual learning. ARiViT merges vision transformers, convolutions, and adversarial learning to tackle the complexities of brain tumor detection. ARiViT’s core generator integrates RiT blocks, merging residual convolutions and transformers, with residual connections, channel compression, and strategic weight sharing for enhanced capabilities and computational efficiency. ARiViT demonstrates exceptional robustness in handling noisy images and adapts well to various modality setups. Our systematic approach involves dataset splitting into training, testing, and validation sets, maintaining distributions of 80:20 and subsequently 70:30, ensuring rigorous evaluation. Additionally, our data preprocessing strategies are tailored to handle noisy and low-quality images, as well as those with unusual tumor shapes. The GAN architecture embedded in ARiViT effectively manages the complexities inherent in such image variations, ensuring reliable tumor detection even in challenging scenarios. Through exhaustive comparative evaluations, ARiViT proves its superiority over existing CNN and transformer techniques, as evidenced by both qualitative and quantitative analyses. Our efforts culminate in remarkable achievements, including an F1-score of 98.09%, establishing a pioneering solution in the demanding field of brain tumor detection.