Generative Deep Learning Methods for Improving Few-Shot Segmentation of Infrared Images

Abstract

Image semantic segmentation is an essential topic in computer vision. However, most current deep learning-based networks primarily focus on large-scale visible spectrum datasets, specifically RGB images. Consequently, these models often struggle to perform well on segmentation in adverse environmental conditions such as total darkness, poor illumination, and smog, etc. On the other hand, a thermal infrared (IR) imaging system overcomes the limitations of visible spectrum imaging. Leveraging the advantages of thermal IR imaging, semantic segmentation using infrared images has extensive real-world applications, including defense applications, autonomous driving, and medical imaging. However, there remain challenges in harnessing the benefits of IR images. Firstly, accessing large-scale annotated IR images is challenging due to security considerations. Secondly, we can face data with unseen classes or rare categories in various IR imaging applications. Lastly, thermal IR images typically exhibit low-resolution, low-contrast, and obscure object boundaries due to the nature of thermal cameras. These images are composed of a single grayscale channel, which provides limited information compared to RGB images. Thus, applying only IR images to existing semantic segmentation models leads to inaccurate performance. In this thesis, our aim is to improve existing few-shot segmentation models to enable robust few-shot segmentation of IR images. To this end, we propose the use of generative methods to enhance our few-shot segmentation model by generating two types of synthesized data: one for augmenting the training data and the other for providing conditioning information. Results show that these strategies substantially improve few-shot segmentation of IR images.