ESTRO 2025 - Abstract Book

S3424

Physics - Machine learning models and clinical applications

ESTRO 2025

Results: The top five highest scoring values for every individual O, S, D, risk score, and the linked failure modes are illustrated in Figure 2. The highest identified risks relate to inadequate review and quality of the auto segmentation and were dominated by a low D. Treating the wrong patient within an automated workflow had a very high S, but both O and D were perceived as low. Another high S was given to the incorrect prescription assignment, leading to an incorrect run of the FAW. The highest overall risk scores were attributed to suboptimal manual review and suboptimal protocols applied during the FAW. These carried lower S scores, but were expected to be harder to detect and to occur more frequently. Both the quantitative analysis of the scores and qualitative analysis of the notes confirmed the above.

Conclusion: The FMEA analysis highlighted wariness in the consistency of people dealing with automation rather than lack of trust in the FAW itself. Major concerns were the ability of people to correctly judge output in case of low generalizability of the model and increasing skill degradation. Consequently, tools and education are needed to help users interpret FAW output before large scale clinical application is feasible.

Keywords: FMEA, automated workflows

References: 1 McIntosh C et al. Clinical integration of machine learning for curative-intent radiation treatment of patients with prostate cancer. Nat Med 2021;27:999 – 1005. https://doi.org/10.1038/s41591-021-01359-w. 2 Gooding M et al. Fully automated radiotherapy treatment planning: A scan to plan challenge. Radiotherapy and Oncology 2024; Volume 200, 110513. https://doi.org/10.1016/j.radonc.2024.110513. 3 Nealon KA et al. Using Failure Mode and Effects Analysis to Evaluate Risk in the Clinical Adoption of Automated Contouring and Treatment Planning Tools. Pract Radiat Oncol 2022;12:e344 – 53. https://doi.org/10.1016/j.prro.2022.01.003.