ESTRO 2024 - Abstract Book

S4454

Physics - Machine learning models and clinical applications

ESTRO 2024

Threshold level

Sensitivity

-

Specificity

-

Accuracy - Test Data [%]

Sensitivity - Test Data [%]

Specificity - Test Data [%]

Accuracy - Test Data [%]

Test Data [%]

Test Data [%]

0.5 0.6 0.7

81.25 86.67 85.71 84.62

100 100

83.33 88.89 83.33 77.78 77.78

76.47 81.25 92.31 92.31

100 100

77.78 83.33 88.89 88.89 88.89 72.22

92.31 84.62 69.23 30.77

92.31 92.31 84.62 61.54

0.75

0.8 0.9

100 100

100 100

50

We selected a conservative threshold level = 0.8. Anyway, a threshold = 0.75 could intercept all plans with gamma <85% for both models (Figure 2).

For XGBoost, we evaluated model stability by selecting 10 different random seeds. Results averaged over 10 random seeds (threshold level = 0.8) are AUC 10 = 0.8±0.1, Sensitivity 10 = 91±3.9, Specificity 10 = 78.5±9, Accuracy 10 = 78.9±7.4. XGBoost could correctly identify all plans with gamma <85% for all random seeds analysed.

Conclusion: