Deep generative models for phase retrieval in computational microscopy

Abstract

Computational imaging is an important tool in science and medicine. It enables us to observe nano scale phenomenon which are beyond the range of any lens-based optical setup. We here tackle the problem of phase retrieval which occurs in a plethora of imaging tasks such as computational microscopy, X-ray crystallography and astronomy, among others. Better solutions to the phase retrieval problem would expand the capabilities of such imaging tools to observe physical/biological phenomenon which are currently not possible to observe. The importance of the phase retrieval problem cannot be understated. And yet it has proved to be a very challenging problem. The need for a reliable, practical and efficient algorithm is felt even after decades of research effort directed towards understanding the physics/mathematics of phase retrieval and developing principled solutions for it. Specifically, in this report, we focus on the phase retrieval for computational microscopy. Computational microscopes are a crucial tool in understanding disease progression, virus behaviours and developing new vaccines. Recently, deep learning methods have garnered a lot of attention because of their surprising success at solving ill-posed problems where most of the iterative methods or traditional machine learning methods fail to perform satisfactorily. Subsequently, there has been a lot of effort to see if deep learning can be used to improve the state-of-the-art on phase retrieval. But surprisingly, most of these developments are yet to receive wide-spread acceptance in practice. This work focuses on developing practical deep learning methods for phase retrieval. We propose datasets which can evaluate the suitability of existing methods on practical phase retrieval data. Based on this, two new methods are proposed for practical phase retrieval and finally, its benefits are demonstrated on a real-world problem of bio-medical image reconstruction. The first part of this thesis considers the end-to-end deep learning approach for phase retrieval.We highlight a fundamental difficulty for learning that previous work has neglected, likely due to the use of popular computer vision datasets (such as MNIST, Imagenet etc.). We propose a simple modification over the datasets such that they reflect the difficulty of practical phase retrieval settings. Experiments on these datasets confirm the well-known intuition that the difficulty of phase retrieval is due to symmetries present in the forward function. And without a careful symmetry breaking of the data, even end-to-end learning suffers from the same failures as most traditional methods. These new datasets form a benchmark which can be used to evaluate existing end-to-end learning methods and those proposed in the next section. The next section focuses on developing a new end-to-end learning framework which can perform phase retrieval without the need for a separate symmetry-breaking procedure. By incorporating the physics of problem into the end-to-end learning framework, we propose a simple yet different formulation for PR which can automatically perform symmetry-breaking. Experiments demonstrate that this new method consistently returns better qualitative results as compared to a naive application of end-to-end learning. Additionally, it is found that the reconstruction returned by this method serves as a very useful initialization for the traditional iterative approaches which subsequently produce high quality images, The third work focuses on developing a practical method(LoDIP) for low-light computational microscopy. The two major challenges here are the absence of a suitable dataset for the task and high noise conditions due to imaging in low-light. Low-light imaging is important tool in bio-medical imaging as most biological samples are susceptible to radiation damage. Yet, imaging in low-light conditions is difficult due to the increased presence of Poisson noise. Unlike the earlier chapters, this chapter focuses on developing a method for singe-image phase retrieval setting. By combining improvements to the computational algorithm as well as the imaging setup, the proposed method is able to provide better resolution than existing approaches in the low-light regime. Finally, in the last chapter, we propose to extend this new technique to work on practical settings for computational microscopy. We identify a suitable data which can be used to demonstrate the efficacy of the proposed method to practical data.