ESTRO 2021 Abstract Book

S403

ESTRO 2021

relative bias in the observers. Results

The human and AI contours showed a mean Dice similarity coefficient of 0.75±0.12 and Hausdorff distance of 0.44±0.09. Observer’s contour-origin-assignment was correct for 923 cases (68.5%). The number of cases scored ≥4 (not requiring editing) were 332 (194 AI and 138 human, 40%) for “AI” and 449 (120 AI and 329 human, 87.2%) for “human” contours based on observer belief. The ratings of “human” were significantly higher than “AI” contours in both AI (0.95, 95% CI= 0.79,1.11, p <0.001) and human (0.95, 95% CI= 0.82,1.07, p <0.001). The regression analysis performed demonstrated that observers consistently ranked OARs believed to be human 1.00 (95%CI = 0.91-1.09) higher than the OARs believed to be of AI origin (p <0.001). In a subgroup of “incorrectly” predicted cases, most OARs differed significantly in their ratings (Table 1 and Figure 1) except lips and eyes. Table 1. Differences in “Human” vs. “AI” clinical acceptability ratings for “incorrectly” predicted contours OAR (95% CI) p value Acoustic 0.72 (0.47, 0.96) <0.01 Brachial plexux 1.0 (0.59, 1.41) <0.01 Chiasm 1.61 (0.81, 2.4) <0.01 Eye 0.4 (-0.03, 0.83) 0.069 Lens 0.67 (0.3, 1.05) <0.001 Lips 0.45 (-0.34, 1.24) 0.24 Optic nerve 0.58 (0.12, 1.04) <0.016 Parotid 1.29 (0.77, 1.8) <0.001 Spinal cord 0.81 (0.12, 1.51) <0.026

Figure 1. Violin plot of clinical acceptability ratings for "incorrectly" predicted contours

Conclusion Our results suggest that significant observer bias exists towards human or against AI when assigning clinical acceptability scores of OAR delineation. Future attempts to assess clinical acceptability of AI/human generated contours should incorporate observer blinding and anti-biasing strategies to determine objective metrics of clinical acceptability. OC-0519 Feasibility of local proton-photon plan comparison in a national proton trial in head-neck cancer J. Friborg 1 1 Rigshospitalet, Department of Oncology, Copenhagen, Denmark Purpose or Objective Due to the specific energy deposition, proton therapy offers theoretical advantages in reducing morbidity from head and neck cancer radiotherapy compared to photons. However, study designs are challenging as not all patients may benefit from proton treatment and simple patient- or tumour characteristics for patient