Advancing Education, Research, and Quality of Care for the Head and Neck oncology patient.
Introduction: Melanoma is known for its rapid growth and metastatic potential. Early and accurate diagnosis can significantly improve outcomes with timely intervention. Recent advancements in artificial intelligence (AI), particularly the new generation of automated machine learning (AutoML), offer the potential for enhanced diagnostic accuracy with fewer barriers to implementation. This study evaluates Google Vertex AI's AutoML platform for the automated classification of melanoma, nevus, and seborrheic keratosis using image-based analysis.
Methods: We utilized the Kaggle melanoma dataset from the International Skin Imaging Collaboration (ISIC), comprising 1,597 dermoscopic images: 462 seborrheic keratoses, 471 melanomas, and 664 nevi. The dataset was divided into 80% training, 10% validation, and 10% test sets. We employed Google Cloud Vertex AI's AutoML platform to train a 2D image classifier specifically designed for image classification, enabling the automated categorization of skin lesions into three groups: melanoma, nevus, and seborrheic keratosis.
Results: The model achieved an accuracy of 83% in the correct classification of all lesion types. For melanoma, at a confidence threshold of 0.5, the model demonstrated a recall of 82.4% and a precision of 88.6%. The specificity and sensitivity for melanoma were 72% and 82.4%, respectively, with a positive predictive value (PPV) of 81% and a negative predictive value (NPV) of 89%. The area under the curve (AUC) was 0.90, indicating strong model performance in distinguishing melanoma from benign lesions. Additionally, the F1 score for melanoma classification was 0.82, reflecting a balanced precision and recall for accurate diagnosis. For nevi, the PPV was 81.2% with an accuracy of 90.9%. For seborrheic keratosis, the PPV was 91.4% with an accuracy of 84.8%.
Conclusion: Using Vertex AI’s AutoML, we developed an advanced deep-learning model for rapid and noninvasive differentiation between melanomas and benign lesions, requiring minimal programming. AutoML features, such as transfer learning and hyperparameter optimization—make it an accessible and scalable tool for clinical use. This streamlined workflow enables head and neck oncologists to distinguish between melanoma and other skin lesions with high precision, facilitating in-office diagnostic support.
Notably, previous studies on similar datasets have applied various machine learning algorithms for melanoma detection, with reported accuracy rates ranging from 75% to 87%, often requiring specialized coding skills and expertise in machine learning. In comparison, our model achieved an accuracy rate of 83%, closely matching the diagnostic accuracy observed in oncologists with approximately two years of training, who typically reach between 75% and 85% accuracy. Our findings, achieved through a no-code approach, perform comparably to these methods, suggesting that with access to appropriate local data, future clinical AI models could be developed directly by healthcare professionals without technical expertise. This study highlights AutoML's potential to shift melanoma screening practices toward real-time, reliable, and user-friendly diagnostic tools for the clinical setting. These results need to be validated in larger datasets and compared to the overall accuracy of experts in distinguishing between melanoma and non-melanoma lesions.