ESTRO 2023 - Abstract Book

S351

Sunday 14 May 2023

ESTRO 2023

1 Therapanacea, Artificial Intelligence, Paris, France; 2 Gustave Roussy, Paris-Saclay University, Inserm 1030, Molecular Radiotherapy and Therapeutic Innovation, Villejuif, France; 3 Centre Léon Bérard, Radiation Oncology, Lyon, France; 4 Ecole des Ponts, Université Gustave Eiffel, CNRS, Laboratoire d'Informatique Gaspard-Monge, Marne-la-Vallée, France Purpose or Objective Co-registration between in vivo radiological imaging and ex vivo histopathologic Whole Slide Image (WSI) enables pixel- wise mapping of ROIs. It brings biological insights for radiation oncologists towards new guidelines and homogenization of clinical practice across centers. In addition, AI-based segmentation methods often fail to learn ROIs on radiology due to interobserver contour variability and can benefit from these cross-modality ground truth labels. However, in addition to the differences in resolution scales, color intensities, and nature of data (2D vs. 3D), the task is often manual and challenging because the tissue undergoes severe deformation and shrinkage during the histological process. We propose a cutting-edge deep-learning framework for the automatic registration of 2D histopathology with 3D radiology. The pipeline is generalizable to any radiological modality and allows the reconstruction of synthetic 3D histology. Materials and Methods We collected a cohort from 77 patients (joint collaboration Gustave Roussy Institute - Centre Léon Bérard, France) on whom were acquired both pre-operative H&N 3D CT scans and 4 to 11 digitalized 2D WSIs after total laryngectomy. Our novelty is two-fold: first, to solve the multimodal issue, we developed a transfer model based on cycleGAN to bring both images to the same modality. Second, for the 2D-3D problem, we built a slice-to-volume unsupervised registration pipeline. Both blocks are trained in parallel for mutual benefit (Figure 1). Moreover, correct mapping is difficult to achieve without proper initialization because of the tissue artifacts mentioned above. To overcome it, we guided the registration by rigidly aligning both thyroid and cricoid cartilages which are supposedly not distorted during tissue preparation procedures.

Results Figure 2 highlights some visual samples. The deformed WSI (c) is closely aligned to the fixed CT scan (b), for rigid regions like cartilage as well as for soft tissue at the edge of slide inclusion. Quantitatively for all patients, we report a mean Dice Score of 0.91/1 between cartilage masks and a mean Normalized Cross Correlation of 0.89/1 across all tissue. For the modality transfer generation, we can reconstruct synthetic images with a Structure Similarity index of 0.82/1, enabling a 3D reconstruction of the histology specimen by filling empty slices with generated histology from CT.

Conclusion