The Radiation Oncology Feed For radiation oncologists · physicists · RTTs · dosimetrists
technical-relevance AI cardiac-toxicity

Large language model framework extracted cardiac events with up to 85.5% accuracy

The framework achieved 85.5% external-validation accuracy and reduced estimated review time from 822 hours to 2.3–4.8 hours.

RadOncToday · ·1 min read

KEY POINTS

  • This retrospective study included 411 patients with breast or lung cancer treated with radiotherapy at two institutions; physician review of the complete electronic health record provided the reference standard.
  • The TRACER framework combined structured problem-list screening with locally deployed large language models analysing filtered, temporally classified excerpts from unstructured clinical notes.
  • The cohorts comprised 178 lung cancer patients for development, 88 breast cancer patients for internal validation and 145 lung cancer patients for external validation.
  • DeepSeek-R1 achieved 85.2% accuracy in the internal breast cancer cohort, while Llama-3.3 achieved 85.5% accuracy on external institutional validation. Development-cohort accuracy reached 83.4%, compared with 73.1% for structured problem-list matching and 47.8% for zero-shot classification.
  • Automated processing required 2.3–4.8 hours for all 411 patients, versus an estimated 822 person-hours for manual review, representing an approximately 170-fold reduction in time burden.

CLINICAL TAKEAWAY

TRACER could make large retrospective cardiotoxicity studies more feasible by extracting cardiac events from fragmented structured and narrative records. However, the framework was designed for cohort discovery rather than clinical decision support, and its classification errors, task-specific definitions and need for human verification make the evidence technically relevant rather than practice-changing.

SOURCE

International Journal of Radiation Oncology, Biology, Physics