Riemannian Manifold-Based Modeling and Classification Methods for Video Activities with Applications to Assisted Living and Smart Home

Doctoral thesis, 2016

This thesis mainly focuses on visual-information based daily activity classification, anomaly detection, and video tracking through using visual sensors. The main reasons for adopting visual-information based methods are due to: (i) vision plays a major role in recognition/classification of activities which is a fundamental issue in a human-centric system; (ii) visual sensor-based analysis may possibly offer high performance with minimum disturbance to individuals' daily lives. Manifolds are employed for efficient modeling and low-dimensional representation of video activities, due to the following reasons: (a) the nonlinear nature of manifolds enables effective description of dynamic processes of human activities involving non-planar movement, which lie on a nonlinear manifold other than a vector space; (b) many video features of human activities may be effectively described by low-dimensional data points on the Riemannian manifold while still maintaining the important property such as topology and geometry; (c) the Riemannian geometry provides a way to measure the distances/dissimilarities between different activities on the nonlinear manifold, hence is a suitable tool for classification and tracking. In this thesis, six different methods for visual analysis of human activities are introduced, including fall detection in video, activity classification in image and video, and video tracking using single camera and multiple cameras. Considering the contribution in theoretical aspects, the use of Riemannian manifolds was investigated for mathematical modeling of video activities, and new methods were developed for characterizing and distinguishing different activities. Experiments on real-world video/image datasets were conducted to evaluate the performance of each method. Results, comparisons, and evaluations showed that the methods achieved state-of-the-art performance. From the perspective of application, the methods have a wide range of potential applications such as assisted living, smart homes, eHealthcare, smart vehicles, office automation, safety systems and services, security systems, situation-aware human-computer interfaces, robot learning, etc.