Clinical and dermoscopy image-based deep learning models for skin lesion diagnosis in clinical practice

Dermoscopic images have been the main driver for model developments in skin lesion classification and segmentation in the past few decades. The performance of state-of-the-art deep learning models has been reported on diverse skin lesion diagnostics tasks. The promising results are often reported in controlled settings where models are developed on training datasets comprising 70–90% of the study datasets and the model’s performance is evaluated on the remaining 10-30% testing sets. The need for mobile diagnostic tools powered by smartphone applications necessitates research into the performance of deep learning models developed from dermoscopic, clinical, or multimodal images in realistic scenarios featuring out-of-domain datasets outside of the distribution of the original training set. The performance of four pre-trained deep learning architectures is investigated in this study using clinical and dermoscopic lesion images. Binary and multiclass classification experiments were conducted with dermoscopic and clinical lesion images from the ISIC2019 and PADUFES20 datasets. Contrary to earlier studies, the performance of the resulting models was evaluated on an out-of-distribution dataset featuring dermoscopic and clinical images in realistic settings. The results in this study suggest that models developed from clinical images generalize better to out-of-distribution clinical and dermoscopic images than models developed from dermoscopic images.