ESTRO 2023 - Abstract Book

S583

Monday 15 May 2023

ESTRO 2023

algorithms. However, to validate such contouring algorithms, guidelines are needed, and quantification of inter-expert observer variance can help to comprehend the quality of automated contours. The project aimed to quantify the inter-observer variance of OAR in the head and neck region across DAHANCA oncology experts. Materials and Methods The DAHANCA OAR contouring guidelines were clarified and discussed through a two-day workshop. The discussions were based on a pre-workshop national contouring audit where 20 expert oncologists contoured 17 OARs of the same patient. Subsequently, 15 experts were asked to contour 26 patients at the workshop to collect multiple contours on the same patient. For each patient, all contours were compared in pairs using the Dice index, mean surface distance (MSD), and 95% Hausdorff distance. To illustrate 3D spatial areas of larger contouring variance, all OAR were deformable registered to a reference patient, and the average contouring variance was displayed (fig 1).

Results In total, 3545 OAR were delineated in 26 patients, 17 per patient. The median number of observers per organ was 9 (IQR 7-9). The most familiar organs, like the brain stem, oral cavity, parotid and submandibular glands, esophagus and thyroid, showed high median dice, above 0.8. The most recent guidelines with new OAR, like pharynx constrictor muscles (PCM), lips, and buccal mucosa, had the lowest median dice, below 0.6. The structures with the highest tissue contrast, like submandibular and thyroid glands, showed median MSD of just above 1 mm, while most other OAR had median MSD below 3 mm (fig 2). The spinal cord showed a very large variance linked to observers' inconsistency in following the guidelines of contouring the full length in the caudal direction. Most information on inter-observer variation is found in the variance plots (fig 1), from which it is clear that the deep part of the parotid gland had the highest uncertainty. The transition between PCM low and esophagus was not well defined. Likewise, there is roughly one CT slice of variance for the transition between the glottic larynx and supraglottic larynx.