Browsing by Author "Shen, Jianhong"

Now showing 1 - 15 of 15

Bayesian video dejittering by BV image model
(2002-12) Shen, Jianhong
Line jittering, or random horizontal displacement in video images, occurs when the synchronization signals are corrupted in video storage media, or by electromagnetic interference in wireless video transmission. The goal of intrinsic video dejittering is to recover the ideal video directly from the observed jittered and often noisy frames. The existing approaches in the literature are mostly based on local or semi-local filtering techniques and autoregressive image models, and complemented by various image processing tools. In this paper, based on the statistical rationale of Bayesian inference, we propose the first variational dejittering model based on the bounded variation (BV) image model, which is global, clean and self-contained, and intrinsically combines dejittering with denoising. The mathematical properties of the model are studied based on the direct method in Calculus of Variations. We design one effective algorithm and present its computational implementation based on techniques from numerical partial differential equations (PDE) and nonlinear optimizations.
Digital inpainting based on the Mumford-Shah-Euler image model
(2001-07) Esedoglu, Selim; Shen, Jianhong
Image inpainting is an image restoration problem, in which image models play a critical role, as demonstrated by Chan, Kang, and Shen's recent inpainting schemes based on the bounded variation and the elastica image models. In the present paper, we propose two novel inpainting models based on the Mumford-Shah image models and the its high order correction -- the Mumford-Shah-Euler image model. We also present their efficient numerical realization based on the Gamma-convergence approximations of Ambrosio and Tortorelli, and De Giorgi.
Euler's elastica and curvature based inpaintings
(2001-04) Chan, Tony F.; Kang, Sung Ha; Shen, Jianhong
Image inpainting is a special image restoration problem for which image prior models play a crucial role. Euler's elastica was first introduced by Mumford [21] to computer vision as a prior curve model. By functionalizaing the elastica energy, Masnou and Morel [19] proposed an elastica based variational inpainting model. The current paper is intended to contribute to the development of its mathematical foundation, and the study of its properties and connections to the earlier works of Bertalmio, Sapiro, Caselles, and Ballester [2] and Chan and Shen [6,7]. A computational scheme based on numerical PDEs is presented, which allows the handling of topologically complex inpainting domains.
A good image model eases restoration - on the contribution of Rudin-Osher-Fatmi's BV image model
(2002-02) Chan, Tony F.; Shen, Jianhong
What we believe images are determines how we take actions in image and low-level vision analysis. In the Bayesian framework, it is known as the importance of a good image prior model. This paper intends to give a concise overview on the vision foundation, mathematical theory, computational algorithms, and various classical as well as unexpected new applications of the BV (bounded variation) image model, first introduced into image processing by Rudin, Osher, and Fatemi in 1992 [Physica D, 60:259-268].
Morphologically invariant PDE inpaintings
(2001-04) Chan, Tony F.; Shen, Jianhong
This paper studies the PDE method for image inpaintings. Image inpainting is essentially an image interpolation problem, with wide applications in film and photo restoration, text removal, special effects in movies, disocclusion, digital zoom-in, and edge-based image compression and coding. Bertalmio, Sapiro, Caselles, and Ballester (2000) [3] first innovatively introduced the PDE method for the inpainting problem. Ever since, the authors of the present paper have worked along this line and developed the PDE method, mostly inspired by the Bayesian and variational method (especially by good image {\rm prior} models). The current paper has two major goals. First, by surveying all the recent PDE inpainting techniques, we intend to develop a unified viewpoint based on two infinitesimal mechanisms: transportation and curvature driven diffusions (CDD). Furthermore, based this knowledge, we construct a new class of third order inpainting PDEs, which is derived from the set of axioms (or principles) refined from the existing works: morphological invariance, rotational invariance, stability principle, and linearity principle.
Multiphase image segmentation via Modica-Mortola phase transition
(2006-06) Jung, Yoon Mo; Kang, Sung Ha; Shen, Jianhong
Non-texture inpainting by curvature-driven diffusions (CDD)
(2001-02) Chan, Tony F.; Shen, Jianhong
Inpainting refers to the practice of artists of restoring ancient paintings. Simply speaking, inpainting is to complete a painting by filling in the missing informa tion on prescribed domains. On such domains, the original painting has been damaged due to aging, scratching, or some other factors. Inpainting and disocclusion in vision analysis are closely connected but also clearly different. Both try to recover the missing visual information from a given 2-D image, and mathematically, can be classified into the same category of inverse problems. The difference lies in both their goals and approaches. The main goal of disocclusion is to model how human vision works to complete occluded objects in a given 2-D scene, and understand their physical or ders in the direction perpendicular to the imaging plane, and thus reconstruct approximately a meaningful 3-dimensional world (Nitzberg, Mumford, and Shiota [14]). The outputs from disocclusion are complete objects, and their relative orders or depth. Inpainting, on the other hand, is to complete a 2-D image which have certain regions missing. The output is still a 2-D image. (In applications, a missing region can indeed be the 2-D projection of a real object, such as the female statue in Figure 9.) Therefore, from the vision point of view, inpainting is a lower level process compared to disocclusion. This fundamental difference naturally influences the approaches. The main approach for disocclusion is to segment the regions in a 2-D image, and then logically connect those which belong to the projection of a same physical object, and finally generate the order or depth for all the completed objects. Edge completion is one crucial step during the whole process. Disocclusion also often uses some high level information about objects (such as the near symmetry of human faces). For inpainting, an ideal scheme should be able to reconstruct an incomplete 2-D image in every detail so that it looks "complete" and "natural." More specifically, to inpaint, is not only to complete the broken edges, but also to connect each broken isophote (or level-line), so that the 2-D objects completed in such a way show their natural variation in intensity (or color for color images) [3, 6,11]. This comparison helps us understand better the real nature of the inpainting problem in a broader context. The terminology of digital inpainting was first introduced by Bertalmio, Sapiro, Caselles, and Ballester [3]. Inspired by the real inpainting process of artists, the authors invented a successful digital inpainting scheme (referred to below as the BSCB inpainting scheme for convenience) based on the PDE method. The authors also deepened the interest in digital inpainting by demon strating its broad applications in text removal, restoring old photos, and creating special effects such as object disappearance from a scene. Though a qualitative understanding based on the transportation mechanism can be well established, rigorous mathematical analysis on the BSCB scheme appears to be much more difficult. This has encouraged Chan and Shen [6] to develop a new inpainting model which is founded on the variational principle. Since the energy function is based on the total variational (TV) norm [6], the model is called TV inpainting. The TV inpainting scheme is surprisingly a close variation of the well known restoration model of Rudin, Osher and Fatemi (16, 17].
On some quantum and analytical properties of fractional Fourier transforms
(2002-05) Shen, Jianhong
Fractional Fourier transforms (FrFT) are a natural one-parameter family of unitary transforms that have the ordinary Fourier transform embedded as a special case. In this paper, following the efforts of several authors, we explore the theory and applications of FrFT, from the standpoints of both quantum mechanics and analysis. These include the phase plane interpretation of FrFT, FrFT's role in the order reduction of certain classes of differential equations, the integral representation of FrFT, and its Paley-Wiener theorem and Heisenberg uncertainty principle. Our two major tools are quantum operator algebra and asymptotic analysis such as the singular perturbation theory and the stationary phase technique.
On the foundations of vision modeling III. Pattern-theoretic analysis of Hopf and Turing's reaction-diffusion patterns
(2003-05) Shen, Jianhong; Jung, Yoon Mo
After Turing's ingenious work on the chemical basis of morphogenesis fifty years ago, reaction-diffusion patterns have been extensively studied in terms of modelling and analysis of pattern formations (both in chemistry and biology), pattern growing in complex laboratory environments, and novel applications in computer graphics. But one of the most fundamental elements has still been missing in the literature. That is, what do we mean exactly by (reaction-diffusion) {\em patterns}? When presented to human vision and visual system, the patterns usually look deceptively simple and are often tagged by household names like {\em spots} or {\em stripes}. But are such split-second pattern identification and classification equally simple for a computer vision system? The answer does not seem to be confirmative, just as in the case of face recognition, one of the greatest challenges in contemporary A.I. and computer vision research. Inspired and fuelled by the recent advancement in mathematical image and vision analysis (Miva), as well as modern {\em pattern theory}, the current paper develops both statistical and geometrical tools and frameworks for identifying, classifying, and characterizing common reaction-diffusion patterns and pattern formations. In essence, it presents a data mining theory for the scientific simulations of reaction-diffusion patterns.
On the foundations of vision modeling IV. Weberized Mumford-Shah model with Bose-Einstein photon noise: Light adapted segmentation inspired by vision psychology, retinal physiology, and quantum statistics
(2003-12) Shen, Jianhong; Jung, Yoon-Mo
Human vision works equally well in a large dynamic range of light intensities, from only a few photons to typical midday sunlight. Contributing to such remarkable flexibility is a famous law in perceptual (both visual and aural) psychology and psychophysics known as Weber's Law. There has been a great deal of efforts in mathematical biology as well to simulate and interpret the law in the cellular and molecular level, and by using linear and nonlinear system modelling tools. In terms of image and vision analysis, it is the first author who has emphasized the significance of the law in faithfully modelling both human and computer vision, and attempted to integrate it into visual processors such as image denoising ( Physica D, 175, pp. 241-251, 2003). The current paper develops a new segmentation model based on the integration of both Weber's Law and the celebrated Mumford-Shah segmentation model ( Comm. Pure Applied Math., 42, pp. 577-685, 1989). Explained in details are issues concerning why the classical Mumford-Shah model lacks light adaptivity, and why its ``weberized" version can more faithfully reflect human vision's superior segmentation capability in a variety of illuminance conditions from dawn to dusk. It is also argued that the popular Gaussian noise model is physically inappropriate for the weberization procedure. As a result, the intrinsic thermal noise of photon ensembles is introduced based on Bose and Einstein's distribution in quantum statistics, which turns out to be compatible with weberization both analytically and computationally. The current paper then focuses on both the theory and computation of the weberized Mumford-Shah model with Bose-Einstein noise. In particular, Ambrosio-Tortorelli's Gamma-convergence approximation theory is adapted (Boll. Un. Mat. Ital., 6-B, pp. 105-123,1992), and stable numerical algorithms are developed for the associated pair of nonlinear Euler-Lagrange PDEs. Numerical results confirm and highlight the light adaptivity feature of the new model.
On the foundations of vision modeling V. Noncommutative monoids of occlusive preimages
(2004-04) Shen, Jianhong
On the foundations of vision modeling. II. Mining of mirror symmetry of 2-D shapes
(2002-12) Shen, Jianhong
Vision can be considered as a feature mining problem. Visually meaningful features are often geometrical, e.g., boundaries (or edges), corners, T-junctions, and symmetries. Mirror symmetry or near mirror symmetry is very common and useful in image and vision analysis. The current paper proposes several different approaches to extract the symmetry mirrors of 2-dimensional (2-D) mirror symmetric shapes. Proper mirror symmetry metrics are introduced based on Lebesgue measures, Hausdorff distance, and lower-dimensional feature sets. Theory and computation of these approaches and measures are studied.
Variational image inpainting
(2002-07) Chan, Tony F.; Shen, Jianhong
Inpainting is an image interpolation problem, with broad applications in image and vision analysis. This paper presents our recent efforts in developing inpainting models based on the Bayesian and variational principles. We discuss several geometric image models, their role in the construction of variational inpainting models, and the associated Euler-Lagrange PDEs and their numerical computation.
Variational PDE models in image processing
(2002-08) Chan, Tony F.; Shen, Jianhong; Vese, Luminita
In this article, we intend to give a broad picture of mathematical image processing through one of the most recent and very successful approaches -- the variational PDE method. We first discuss two crucial ingredients for image processing: image modeling or representation, and processor modeling. We then focus on the variational PDE method. The backbone of this article consists of two major problems in image processing -- inpainting and segmentation.
Weber's law and weberized TV restoration
(2002-04) Shen, Jianhong
Most conventional image processors consider little the influence of human vision psychology. Weber's Law in psychology and psychophysics claims that human's perception and response to the intensity fluctuation of visual signals are weighted by the background stimulus, instead of being plainly uniform. This paper attempts to integrate this well known perceptual law into the classical total variation (TV) image restoration model of Rudin, Osher, and Fatemi [Physica D, 60:259-268, 1992]. We study the issues of existence and uniqueness for the proposed Weberized nonlinear TV restoration model, making use of the direct method in the space of functions with bounded variations. We also propose an iterative algorithm based on the linearization technique for the associated nonlinear Euler-Lagrange equation.

University Digital Conservancy

Browse by Author

Browsing by Author "Shen, Jianhong"