ESTRO 2025 - Abstract Book

S3846

Physics - Radiomics, functional and biological imaging and outcome prediction

ESTRO 2025

analytical data to develop robust predictive models in radiotherapy, paving the way for more personalized and effective treatment strategies.

Keywords: Low-Dose Radiotherapy, Machine Learning, COVID-19

References: [1] M. Arenas, B. Piqué, L. Torres-Royo, et al., “Treatment of covid-19 pneumonia with low-dose radiotherapy plus standard of care versus standard of care alone in frail patients: The seor-gicor ipacovid comparative cohort trial,” Strahlentherapie und Onkologie, vol. 199, no. 9, pp. 847–856, 2023. [2] J. J. Van Griethuysen, A. Fedorov, C. Parmar, et al., “Computational radiomics system to decode the radiographic phenotype,” Cancer research, vol. 77, no. 21, e104–e107, 2017.

3820

Digital Poster Multi-modality AI predictor of lung cancer immunotherapy treatment response of Durable Response (DR) Jue Jiang 1 , Helena Yu 2 , Daniel Gomez 3 , Harini Veeraraghavan 1 1 Medical Physics, Memorial Sloan Kettering Cancer Center, New York, USA. 2 Medicine, Memorial Sloan Kettering Cancer Center, New York, USA. 3 Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, USA Purpose/Objective: To predict immunotherapy treatment response in patients with stage III-IV lung locally advanced non-small cell lung cancer by integrating multimodal data consisting of pre-treatment computed tomography (CT), clinical text, and tumor mutational burden (TMB) computed through large scale genomic analysis. Material/Methods: A vision-language model (VLM) to predict immunotherapy response was created by combining a pre-trained 3D vision foundation model to extract visual features from pre-treatment and first on-treatment CT [1] with text embeddings extracted from clinical text (age, smoking status, lines of therapy) with TMB provided as prompts to llama-3 8B large language model (LLM)[2]. The whole pipeline is shown in Figure 1. The vision model was created using self-supervised learning techniques applied to 10,412 unlabeled 3D CT scans. Immunotherapy response consisted of durable clinical benefit (DCB) defined as progression free survival exceeding 6 months after start of therapy. VLM used cross-attention and multi-stage training performed first to align the image with text features and next to predict DCB. As pretrained foundation models were used for both vision and text, requirement of large datasets was reduced. Hence, training was performed using 3-fold stratified cross validation with 153 additional patients. Area under the receiver operating curve (AUROC), specificity and sensitivity were computed.

Figure 1. The pipeline of the proposed multi-modality predictor