Automatic Semantic Segmentation Of Kidney Tumors In Computed Tomography Images

Abstract

Semantic segmentation has emerged as a powerful tool for the computational analysis of medical imaging data, but its enormous need for manual effort has limited its adoption in routine clinical practice. Deep learning methods have begun to achieve impressive automatic semantic segmentation performance for a variety of structures in cross-sectional images, but unlike for large well-defined regions like major organs and bones, the performance on small, poorly-circumscribed structures in unpredictable locations, such as lesions, remains relatively poor. This dissertation presents a series of contributions throughout the machine learning pipeline that allow for unprecedented performance on kidney tumor segmentation. Important among these contributions is (1) the demonstration that deep neural networks for cross-sectional image segmentation are highly sensitive to training set label errors around region boundaries, (2) the development of a novel labeling pipeline which avoids such errors while making efficient use of domain expertise, and (3) the extensive benchmarking of a wide variety of deep learning methods applied to a large scale dataset collected using this pipeline. Taken together, these innovations enable the first fully-automatic semantic segmentation of kidney tumors in computed tomography images with performance that is comparable to human experts. The clinical utility of this capability is demonstrated through two studies presenting segmentation-dependent radiomic analyses of kidney tumors, which help us to uncover the relationship between tumor morphology and patient outcomes: First, through the automation of the R.E.N.A.L. score, and second, through the unprecedented segmentation-based analysis of longitudinal kidney tumor scans. Arising from this work are two highly-regarding machine learning competitions (or "challenges") called KiTS19 and KiTS21 which attracted submissions from hundreds of research teams from across the world. These remain some of the most widely-used benchmarks for medical image segmentation today. While experimental results are primarily presented for the kidney tumor segmentation task, the methods developed and findings presented in this dissertation are broadly applicable to any segmentation task where the target structure is small, poorly-circumscribed, found unpredictable locations, and for which the accurate region identification requires domain expertise which is scarce and expensive.