Advancing Education, Research, and Quality of Care for the Head and Neck oncology patient.
Importance: Generating an accurate pathologic stage is critical in the care of surgically managed head and neck cancer patients but pathology documentation may be incomplete, inconsistent, or lacking. This can negatively impact data abstraction for quality assessment initiatives or cancer registries. Current workflows often rely on manual chart review, which is time-consuming, expensive, and susceptible to errors due to reporting and interpretation biases.
Objectives: Quantify inconsistencies in HPV-positive oropharyngeal squamous cell carcinoma (HPV(+)OPSCC) pathologic staging in pathology reports. Build an AI-Powered tool to streamline and improve the accuracy of HPV(+)OPSCC pathologic staging.
Design: Single institution retrospective cross-sectional study.
Setting: Tertiary care referral center, 2018-2024.
Participants: 381 HPV(+)OPSCC patients treated with intent-to-cure surgical therapy.
Methods: The departmental RedCAP Oropharynx Cancer Registry was queried on July 17, 2024 to identify the above patients. We built an AI-powered system in Google Cloud Platform to extract staging information, using natural language processing. The pipeline, developed using Python 3.10.12, uses a patient's medical record number and date of surgery as input. It extracts relevant clinical documentation and then two large language models, Gemini Pro 1.0-001 and Gemini Flash 1.5-001, extract key staging information (tumor location, dimension, invasion, and number of involved lymph nodes). The pipeline incorporates a coded rule-based logic derived from AJCC 8th Edition guidelines to determine pT and pN stages. To improve output accuracy, the pipeline included quality checks and did not produce an output if they were not passed. Results were then compared to the gold standard manual abstraction within the RedCAP registry. Disagreements between manual-extracted and pipeline-extracted results were reviewed by a team including a senior head and neck surgeon.
Results: Our analysis showed inconsistencies in the original tumor pathologic staging within pathology report: 6% (N=24) of records contained contradictory pT or pN stage assignments; either within the pathology report, or between operative and pathology reports. The number of pipeline outputs that did not pass the internal quality checks was 16 pT and 31 pN staging records. The agreement rate for the pipeline-extracted and manual-extracted staging was 96.4% (352/365) for pT stage and 98.6% (345/350) for pN stage. Discrepancy analysis revealed the pipeline outperformed human abstractors for pT staging in 9 of 13 disagreements and in 5 of 5 disagreements for pN staging, thus making the pipeline’s overall accuracy over 99% for both pT and pN staging. Compared to stages reported in pathology reports, the model demonstrated superior accuracy in 13/15 discrepancies for pT staging, and in 12/12 for pN staging.
Conclusions: This tool offers potential benefits for pathologists, multidisciplinary oncology teams, and data abstractors by automating pathologic staging for surgically treated HPV(+)OPSCC patients; a time-consuming and error prone process. This tool can be deployed to check inconsistencies within the medical record and automatically send an email to providers to check the report and resolve the issue. Output can be automatically populated into a clinical registry. Future work will focus on generalizing this approach to other solid tumor types, inclusive of clinical staging.