Multi-View Hand Tracking using Epipolar Geometry-Based Consistent Labeling for an Industrial Application
Paper in proceeding, 2013

This paper addresses a visual tracking and analysis method for automatic monitoring of an industrial manual assembly process, where each worker sequentially picks up components from different boxes during an assembling process. Automatic surveillance of assembling process would enable to reduce assembling errors by giving early warning. We propose a hand tracking and trajectory analysis method from videos captured by several uncalibrated cameras with overlapping views. The proposed method consists of three modules through single-view hand tracking, consistent labeling across views, and optimal decision from multi-view temporal dynamics. The main novelties of the paper include: (a) target model learning with multiple instances through K-means clustering applied to accommodate different levels of light reflection; (b) optimal criterion for consistent labeling of tracked hands across views, based on the symmetric epipolar distance; (c) backward correction of mis-detection by combining epipolar lines with previously tracked results; (d) a multi-view voting scheme for analyzing hand trajectory using binary hand location maps. Experiments have been conducted on videos by multiple uncalibrated cameras, where a person performs assembly operations. Test results and performance evaluation have shown the effectiveness of this method, in terms of multi-view consistent estimation of hand trajectories and accurate interpretation of component assembly actions.

multi-view hand tracking

consistent labeling

multiple view tracking

hand trajectory analysis

epipolar geometry


Yixiao Yun

Chalmers, Signals and Systems, Signal Processing and Biomedical Engineering

Irene Yu-Hua Gu

Chalmers, Signals and Systems, Signal Processing and Biomedical Engineering

Julien Provost

Chalmers, Signals and Systems, Systems and control

Knut Åkesson

Chalmers, Signals and Systems, Systems and control

Seventh ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC 2013), Oc.29-Nov.1, Palm Springs, California, USA


Areas of Advance

Information and Communication Technology

Subject Categories

Information Science

Signal Processing

Computer Vision and Robotics (Autonomous Systems)



More information

Latest update

3/2/2022 6