Advancing Education, Research, and Quality of Care for the Head and Neck oncology patient.
Importance: Subspecialty care is a cornerstone for patients with complex medical conditions. In the United States, over 100 million referrals are made annually to subspecialists, yet nearly ? of these referrals are never completed. The referral process is largely handled by medical office staff and unstandardized. However, a heterogeneous process may be vulnerable to breakdowns and delays in patient care. Deep neural network models are a type of artificial intelligence that allows complex data processing, without human intervention. Here, we evaluate the accuracy and utility of a Natural Language Processing (NLP) model in triaging head and neck patient referrals.
Objectives: To compare the accuracy of a Natural Language Processing (NLP) model in triaging head and neck patient referrals to subspecialty practice coordinators.
Design, Setting, and Participants: An NLP model was developed to process head and neck referrals. Twenty-two patients referred for specialized head and neck care between March and May 2024 were randomly selected. For each patient, the referring clinical note, pathology report, and/or imaging report were extracted from the electronic medical record and processed through the NLP model. The model’s auto-generated referral necessity and urgency were recorded. These same referrals were independently reviewed by three head and neck practice coordinators, each with a minimum of 2.5 years of experience, who provided their own assessments of referral need and urgency. The accuracy of the model’s recommendations was evaluated by comparing its outputs to both the practice coordinator’s assessments and the attending physician’s clinical notes, which served as the ground truth.
Main Outcomes and Measures: The primary outcomes were accuracy rates in determining the referral necessity and urgency.
Results: The NLP model demonstrated superior performance compared to practice coordinators in assessing the urgency and severity of the incoming head and neck patient referral. Using the attending physician’s clinical note as the ground truth, the NLP model achieved an overall accuracy rate of 71%, compared to 65% for the practice coordinators. Notably, in cases where patients presented with moderate or high suspicion of cancer, the NLP model outperformed human personnel by 26%, achieving a 93% accuracy rate compared to 65% for the coordinators. For patients with benign diagnoses or no pathologies, the NLP model’s accuracy was lower, at 29%, while the practice coordinators achieved an accuracy of 54% accuracy.
Conclusions and Relevance: In conclusion, the NLP algorithm outperformed human personnel in triaging new patient referrals and has the potential to serve as a valuable adjunct in identifying urgent cancer cases. Given the ongoing financial pressures and labor shortages healthcare centers pace in the post-pandemic era, neural network models may provide value by optimizing clinical workflows, improving efficiency, and reducing personnel costs associated with processing referrals. Implementing ML/AI models may help healthcare systems better allocate resources while maintaining high standards of patient care.