Non-texture inpainting by curvature-driven diffusions (CDD)

Abstract

Inpainting refers to the practice of artists of restoring ancient paintings. Simply speaking, inpainting is to complete a painting by filling in the missing informa tion on prescribed domains. On such domains, the original painting has been damaged due to aging, scratching, or some other factors. Inpainting and disocclusion in vision analysis are closely connected but also clearly different. Both try to recover the missing visual information from a given 2-D image, and mathematically, can be classified into the same category of inverse problems. The difference lies in both their goals and approaches. The main goal of disocclusion is to model how human vision works to complete occluded objects in a given 2-D scene, and understand their physical or ders in the direction perpendicular to the imaging plane, and thus reconstruct approximately a meaningful 3-dimensional world (Nitzberg, Mumford, and Shiota [14]). The outputs from disocclusion are complete objects, and their relative orders or depth. Inpainting, on the other hand, is to complete a 2-D image which have certain regions missing. The output is still a 2-D image. (In applications, a missing region can indeed be the 2-D projection of a real object, such as the female statue in Figure 9.) Therefore, from the vision point of view, inpainting is a lower level process compared to disocclusion. This fundamental difference naturally influences the approaches. The main approach for disocclusion is to segment the regions in a 2-D image, and then logically connect those which belong to the projection of a same physical object, and finally generate the order or depth for all the completed objects. Edge completion is one crucial step during the whole process. Disocclusion also often uses some high level information about objects (such as the near symmetry of human faces). For inpainting, an ideal scheme should be able to reconstruct an incomplete 2-D image in every detail so that it looks "complete" and "natural." More specifically, to inpaint, is not only to complete the broken edges, but also to connect each broken isophote (or level-line), so that the 2-D objects completed in such a way show their natural variation in intensity (or color for color images) [3, 6,11]. This comparison helps us understand better the real nature of the inpainting problem in a broader context. The terminology of digital inpainting was first introduced by Bertalmio, Sapiro, Caselles, and Ballester [3]. Inspired by the real inpainting process of artists, the authors invented a successful digital inpainting scheme (referred to below as the BSCB inpainting scheme for convenience) based on the PDE method. The authors also deepened the interest in digital inpainting by demon strating its broad applications in text removal, restoring old photos, and creating special effects such as object disappearance from a scene. Though a qualitative understanding based on the transportation mechanism can be well established, rigorous mathematical analysis on the BSCB scheme appears to be much more difficult. This has encouraged Chan and Shen [6] to develop a new inpainting model which is founded on the variational principle. Since the energy function is based on the total variational (TV) norm [6], the model is called TV inpainting. The TV inpainting scheme is surprisingly a close variation of the well known restoration model of Rudin, Osher and Fatemi (16, 17].