ESTRO 2024 - Abstract Book

S4471

Physics - Machine learning models and clinical applications

ESTRO 2024

a reference model. The performance of this reference model was compared to that of the deep learning model in this dataset. Performance measures used were area under the curve (AUC) and calibration slope and intercept.

Results:

Hyperparameter tuning showed that the key parameters for good and stable performance were a cyclic learning rate and Adam optimizer function, whereas, for instance, augmentation strength and filter size were less of influence. Current results showed improved performance with the deep learning model, with the training set AUC increasing from 0.72 [95% confidence interval (CI): 0.71– 0.73] for the reference model to 0.74 [95% CI: 0.71 – 0.78] for the deep learning model. A larger improvement was observed in the validation set with an AUC of 0.70 [95% CI: 0.66 – 0.75] for the reference model and 0.74 [95% CI: 0.70 – 0.77] for the deep learning model. The unseen test set performance improved from 0.71 [95% CI: 0.70 – 0.72] to 0.72 [95% CI: 0.69 – 0.74]. In addition, the calibration plot of the test set showed visually better and more stable calibration for the deep learning model [intercept = 0.03; slope = 1.14] as for the reference model [intercept = 0.04; slope = 0.94] (Figure 2).