Image Reconstruction and Optical Flow Estimation on Image Sequences with Differently Exposed Frames

Doctoral thesis, 2015

The main objective for digital image- and video camera systems is to reproduce a real-world scene in such a way that a high visual quality is obtained. A crucial aspect in this regard is, naturally, the quality of the hardware components of the camera device. There are, however, always some undesired limitations imposed by the sensor of the camera. For example, the dynamic range of light intensities that the sensor can capture in a given image is much smaller than the dynamic range of common daylight scenes and that of the human visual system. Thus, the scene content in certain regions is not properly captured due to over- or underexposure of the sensor. The dynamic range limitation is addressed by signal processing methods that produce a high dynamic range representation of an original scene by fusing information from a sequence of images. Digital cameras systems, in addition to producing images of high visual quality, are increasingly being used for automatic image analysis tasks, where a computer algorithm analyzes the captured image data and outputs some extracted information. Image analysis results also rely on the use of image data that represents the relevant content of real-world scenes. This thesis is concerned with the opportunities and challenges of high dynamic range imaging, in the contexts of high quality image reconstruction and motion analysis by optical flow estimation. A method is proposed that produces a high dynamic range image and jointly enhances the spatial image resolution by exploiting the fact that the input image sequence provides complementary spatial information of the scene. Key characteristics of the human visual system are taken into account in the problem formulation in order to improve the perceived image quality. In addition, a method is proposed for optical flow estimation in high dynamic range scenarios, that benefits from using image sequences with differently exposed frames as input. The produced motion information can be used in motion analysis applications, including active safety systems in vehicles.