ESTRO 2021 Abstract Book

S362

ESTRO 2021

Conclusion For the four investigated tumor sites including 194 patients, plans generated with the new commercial autoplanning solution had similar dosimetrical quality as those generated with an extensively validated alternative autoplanning system. Differences in non-dosimetric plan parameters were minor. OC-0474 Bias and reporting quality of artificial intelligence models in radiotherapy treatment planning M. Sharabiani 1 , E. Clementel 1 , N. Andratschke 2 , N. Reynaert 3 , W. van Elmpt 4 , C. Hurkmans 5 1 European Organisation for Research and Treatment of Cancer (EORTC) , Radiotherapy quality assurance, Brussels, Belgium; 2 University Hospital Zürich, Department of Radiation Oncology, Zurich, Switzerland; 3 Jules Bordet Institute, Medical Physics Department, Brussels, Belgium; 4 Maastricht University Medical Centre, Department of Radiation Oncology, Maastricht, The Netherlands; 5 Catharina Hospital, Department of Radiation Oncology, Eindhoven, The Netherlands Purpose or Objective The number of studies using artificial intelligence (AI)-based models has increased in recent years, but their clinical application remains a point of contention. The goal of this study was to systematically evaluate the risk of bias (ROB) and reporting quality of AI-based models in radiotherapy treatment planning studies using PROBAST (Prediction model Risk Of Bias ASsessment Tool) and TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) guidelines and report the tailored items in each guideline suitable for AI-based models. Materials and Methods A PubMed search was conducted in January 2021 and updated through March 2021. A combination of keywords were used including: “artificial intelligence” OR “machine learning” OR “deep learning” OR “knowledge- based” AND “radiotherapy” AND “treatment planning”. A total of 659 articles were reviewed for title and abstract. 126 articles were selected for full text review, among which 20 were selected to assess ROB and reporting quality for non-randomised studies. The inclusion criteria for the 20 articles were: recently published (2018 to 2021), published in high impact factor journals and the most cited papers. TRIPOD contains a 22 items checklist (37 individual items), where we identified the items pertaining to predictors not relevant to evaluate AI models. PROBAST contains 20 signaling questions from four domains (participants, predictors, outcomes, analysis), from which we similarly exclude the predictor domain, which is not relevant for AI- treatment planning models. Results The two checklists were first examined for applicability in AI-based algorithms. Some of the items for each guideline required adaptations to suit evaluation of the AI models (e.g. missing data was replaced with suboptimal plans).