Multi-View Hand Tracking using Epipolar Geometry-Based Consistent Labeling for an Industrial Application
Paper i proceeding, 2013

This paper addresses a visual tracking and analysis method for automatic monitoring of an industrial manual assembly process, where each worker sequentially picks up components from different boxes during an assembling process. Automatic surveillance of assembling process would enable to reduce assembling errors by giving early warning. We propose a hand tracking and trajectory analysis method from videos captured by several uncalibrated cameras with overlapping views. The proposed method consists of three modules through single-view hand tracking, consistent labeling across views, and optimal decision from multi-view temporal dynamics. The main novelties of the paper include: (a) target model learning with multiple instances through K-means clustering applied to accommodate different levels of light reflection; (b) optimal criterion for consistent labeling of tracked hands across views, based on the symmetric epipolar distance; (c) backward correction of mis-detection by combining epipolar lines with previously tracked results; (d) a multi-view voting scheme for analyzing hand trajectory using binary hand location maps. Experiments have been conducted on videos by multiple uncalibrated cameras, where a person performs assembly operations. Test results and performance evaluation have shown the effectiveness of this method, in terms of multi-view consistent estimation of hand trajectories and accurate interpretation of component assembly actions.

multi-view hand tracking

consistent labeling

multiple view tracking

hand trajectory analysis

epipolar geometry


Yixiao Yun

Chalmers, Signaler och system, Signalbehandling och medicinsk teknik

Irene Yu-Hua Gu

Chalmers, Signaler och system, Signalbehandling och medicinsk teknik

Julien Provost

Chalmers, Signaler och system, System- och reglerteknik

Knut Åkesson

Chalmers, Signaler och system, System- och reglerteknik

Seventh ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC 2013), Oc.29-Nov.1, Palm Springs, California, USA



Informations- och kommunikationsteknik




Datorseende och robotik (autonoma system)



Mer information

Senast uppdaterat