ESTRO 2024 - Abstract Book

S4972

Physics - Radiomics, functional and biological imaging and outcome prediction

ESTRO 2024

The two MRI sequences were stacked into channels of a four-dimensional tensor to be fed to the image encoder. We further attached the output labels of the segmentation network as auxiliary input information. Following the feature extraction into a 512-dimensional space, the dummy-encoded vector of clinical features was appended. Finally, the multi-modal features were mapped by two linear layers into the scalar output.

We trained our network with the Cox partial log-likelihood [2]:

, where r i = β T X i is the predicted hazard ratio, U is the set of patients with LF, and Ω i is the "at-risk" patients Ω i = {j | t j > t i }. The loss function ranks the predicted hazard ratios anti-concordant to the follow-up times while considering the censoring information. As it was computationally not feasible to process all patients at one time, the loss was optimized with gradient descent in batches, and the Ω i were defined locally per batch.

The dataset was split into training (two centers with 253 patients) and testing cohorts (5 centers with 99 patients). Five-fold cross-validation was used for training and internal validation. Each fold’s model was then tested on the test set.

There was an unbalanced LF event rate of 20%. Thus, the models were trained with batches of size ten using stratified batching to put at least one patient with LF in every batch. As 1 is masked on non-censored cases, the batch distribution is important to ensure the loss value is non-zero. To standardize the image size for batching, we enlarged all cropping bounding boxes to the dimensions of 95x123x108 voxels, which respectively represent the maximum sizes for each dimension observed among the GTVs in the dataset. Furthermore, we used gradient accumulation among four batches to increase the number of samples influencing the network on every weight update.

Results:

In Table 1 our results are shown in comparison with two baseline models: one imaging baseline without our specialized cropping and batching methods and no segmentation outputs or clinical features as inputs and one non imaging baseline model with just clinical features. The results indicate that our methods improved the performance while decreasing the prediction uncertainty.