Visual Object Tracking and Classification Using Multiple Sensor Measurements
Licentiatavhandling, 2013

Multiple sensor measurement has gained in popularity for computer vision tasks such as visual object tracking and visual pattern classification. The main idea is that multiple sensors may provide rich and redundant information, due to wide spatial or frequency coverage of the scene, which is advantageous over single sensor measurement in learning object model/feature and inferring target state/attribute in complex scenarios. This thesis mainly addresses two problems, both exploiting multiple sensor measurement. One is video object tracking through occlusions using multiple uncalibrated cameras with overlapping fields of view, the other is multi-class image classification through sensor fusion of visual-band and thermal infrared (IR) cameras. Paper A proposes a multi-view tracker in an alternate mode with online learning on Riemannian manifolds by cross-view appearance mapping. The mapping of object appearance between views is achieved by projective transformations that are estimated from warped vertical axis of tracked object by combining multi-view geometric constraints. A similarity metric is defined on Riemannian manifolds, as the shortest geodesic distance between a candidate object and a set of mapped references from multiple views. Based on this metric, a criterion of multi-view maximum likelihood (ML) is introduced for the inference of object state. Paper B proposes a visual-IR fusion-based classifier by multi-class boosting with sub-ensemble learning. In our scheme, a multi-class AdaBoost classification framework is presented where information obtained from visual and thermal IR bands interactively complement each other. This is accomplished by learning weak hypotheses for visual and IR bands independently and then fusing them as learning a sub-ensemble. Proposed methods are shown to be effective and have improved performance compared to previous approaches that are closely related, as demonstrated through experiments based on real-world datasets.

sensor fusion

Riemannian manifold

visual pattern classification

multiple view geometry

Visual object tracking


multiple sensor measurement

Room EC, floor 5, Hörsalsvägen 11, Chalmers University of Technology
Opponent: Associate Professor Volker Krüger (Aalborg University Copenhagen, Denmark)


Yixiao Yun

Chalmers, Signaler och system, Signalbehandling och medicinsk teknik, Signalbehandling

Multi-view Face Pose Classification by Boosting with Weak Hypothesis Fusion Using Visual and Infrared Images

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings,; (2012)p. 1949-1952

Paper i proceeding

Multi-Class Ada-Boost Classification of Object Poses through Visual and Infrared Image Information Fusion

Proceedings - International Conference on Pattern Recognition, 21st ICPR 2012, Tsukuba,Japan, 11-15 November 2012,; (2012)p. 2865-2868

Paper i proceeding

Image Classification by Multi-Class Boosting of Visual and Infrared Fusion with Applications to Object Pose Recognition

Swedish Symposium on Image Analysis (SSBA 2013), March 14-15, Göteborg, Sweden,; (2013)p. 4-

Konferensbidrag (offentliggjort, men ej förlagsutgivet)

Multi-View Hand Tracking using Epipolar Geometry-Based Consistent Labeling for an Industrial Application

Seventh ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC 2013), Oc.29-Nov.1, Palm Springs, California, USA,; (2013)p. 6-

Paper i proceeding

Maximum-Likelihood Object Tracking from Multi-View Video by Combining Homography and Epipolar Constraints

6th ACM/IEEE Int'l Conf on Distributed Smart Cameras (ICDSC 12), Oct 30 - Nov.2, 2012, Hong Kong,; (2012)p. 6 pages-

Paper i proceeding

Multi-View ML Object Tracking with Online Learning on Riemannian Manifolds by Combining Geometric Constraints

IEEE Journal on Emerging and Selected Topics in Circuits and Systems,; Vol. 3(2013)p. 12 -

Artikel i vetenskaplig tidskrift


Informations- och kommunikationsteknik






Datorseende och robotik (autonoma system)

R - Department of Signals and Systems, Chalmers University of Technology: R021/2013

Room EC, floor 5, Hörsalsvägen 11, Chalmers University of Technology

Opponent: Associate Professor Volker Krüger (Aalborg University Copenhagen, Denmark)

Mer information