Development of an Artificial Intelligence-Based Diagnostic System Using Dermoscopic Images and Evaluation of the Diagnostic System's Place in Dermatology Residency
Özet
Introduction: Artificial intelligence is defined as endowing machines with
abilities such as learning, problem solving, deriving meaning, and remembering.
Today, artificial intelligence has rapidly developed image processing capabilities. In
dermatology, clinicians diagnose conditions using dermoscopy, a method that involves
analyzing specially taken images of lesions. There are studies on the analysis of
dermoscopic images with artificial intelligence. In some studies, artificial intelligence
has been found to be more successful than experienced specialist doctors in diagnosing
dermoscopic images. However, our literature search to date has not found a study
evaluating the position of artificial intelligence's successful diagnostic capability,
especially in dermatology specialist training. The aim of this study is to demonstrate
the contribution of an artificial intelligence trained with dermoscopic images reflecting
the characteristics of our own patient group to the existing skills of residents.
Methods and Materials: Dermoscopic photos taken for diagnosis and follow up purposes at our hospital between 2013-2023 were evaluated. A dataset consisting
of 4,220 images diagnosed with basal cell carcinoma, squamous cell carcinoma,
melanoma, dysplastic nevus, melanocytic nevus, benign keratoses, actinic keratosis,
dermatofibroma, and vascular lesions was prepared. The ISIC 19 open dataset from
the literature was added to this dataset. An image processing artificial intelligence
algorithm was developed. The dataset obtained was used for the training and testing
of the algorithm. A web application was designed to investigate the effect of artificial
intelligence on the diagnostic accuracy of residents. The study included n=17 research
assistants who had received at least one year of dermoscopy training in our unit's
academic training schedule and had at least one year of clinical experience in
dermoscopy. Participants were asked a total of n=54 dermoscopic image diagnosis
questions through the our application, first answering themselves and then with the
support of artificial intelligence. Data was recorded through the application and the
interaction between dermatology assistants and artificial intelligence was analyzed.
Findings: The dataset was created using n=24,731 (85%) dermoscopic photos
from ISIC19 and n=4,220 (15%) dermoscopic photos from patients at Hacettepe
University (HU), totaling n=28,951 photos. The artificial intelligence was tested with
n=5,910 photos. It was found that the artificial intelligence achieved a diagnostic
accuracy of 0.91 (91%). The balanced accuracy rate of the algorithm was calculated
as 0.78 (78%) and the F1 score as 0.80. According to a 54-question participant
evaluation test prepared from the test set, the change in accuracy rates of the
participants (n=17) with artificial intelligence support was calculated as 0.13
(p<0.001). With artificial intelligence support, the sensitivity increase ranged from a
minimum of 0.01 to a maximum of 0.24 across all diagnoses. The highest improvement
in sensitivity was observed in basal cell carcinoma with 0.24 (p=0.001). This was
followed by melanocytic nevus with 0.20 (p=0.001) and squamous cell carcinoma with
0.19 (p=0.002). The lowest improvement in sensitivity value was calculated as 0.01
(p=0.773) for the diagnosis of dysplastic nevus. In the subgroup analysis according to
experience, first-year research assistants had the highest change in accuracy rates with
artificial intelligence support, increasing by 0.18 (p=0.068). For second-year research
assistants, the increase in accuracy rate with artificial intelligence support was 0.10
(p=0.144); for the third year, it was 0.12 (p=0.068), and for the fourth year, the change
was 0.11 (p=0.042). No significant relationship was found between the increase in
accuracy rate provided by artificial intelligence support and the research assistants'
years of experience.
Conclusion: Our study has shown that artificial intelligence using dermoscopic
images increased the diagnostic accuracy of dermatology residents. Besides, artificial
intelligence support has been observed to increase sensitivity in malignant diagnoses,
including melanoma and especially non-melanoma skin cancers.
