Deep Spectral Spatial Feature Enhancement Through Transformer for Hyperspectral Image Classification

Hyperspectral image (HSI) data has a wide range of spectral information that is valuable for numerous tasks. HSI data encounters some challenges, such as small training samples, data scarcity, and redundant information. Researchers present numerous investigations to address these challenges, with convolutional neural networks (CNNs) being extensively used in HSI classification because of their capacity to extract features from data. Moreover, vision transformers have demonstrated their ability in the remote sensing field. However, the training of these models required a significant amount of labeled training data. We proposed a vision-based transformer module that consists of a multiscale feature extractor to extract joint spectral-spatial low-level, shallow features. For high-level semantic feature extraction, we proposed a regional attention mechanism with a spatially gated module. We tested the proposed model on four publicly available HSI datasets: Pavia University, Salinas, Xuzhou, Loukia, and the Houston 2013 dataset. Using only 1%, 1%, 1%, 2%, and 2% of the training samples from the five datasets, we achieved the best classification in terms of overall accuracy (OA), average accuracy (AA), and Kappa coefficient.