MirrorTablet: Exploring a low-cost mobile system for capturing unmediated hand gestures in remote collaboration
Paper in proceeding, 2017

Direct and natural images of hand gestures have been shown to benefit remote collaboration on physical tasks in several settings, including ad-hoc ones. However, to capture such unmediated hand gestures within collaborative tasks, existing approaches require stationary hardware systems or heavily instrumented mobile devices, making them unfeasible for use at ad-hoc workplaces. We present MirrorTablet, a low-cost mirror-based system taking advantage of the built-in front-facing camera of a tablet to capture the user’s unmediated hand interactions on and above the screen. This system requires minimal instrumentation of the tablet and can be easily (un-)mounted, making it suitable for mobile usage. A user study with ten pairs of participants on a helper-worker setup working on construction tasks showed that MirrorTablet improved task completion time and had positive effects on participants’ perceived workload when working with unfamiliar tasks compared to using a common sketch-only interface. In addition, qualitative feedback yielded design considerations on hand visualization for mobile device remote collaboration on physical tasks.

Hand Interactions

Mobile Devices

Remote Collaboration

Physical Tasks

Hand Gestures


Khanh Duy Le

Chalmers, Applied Information Technology (Chalmers), Interaction design

K. N. Zhu

City University of Hong Kong

Morten Fjeld

Chalmers, Applied Information Technology (Chalmers), Interaction design

ACM International Conference Proceeding Series

978-1-4503-5378-6 (ISBN)

16th International Conference on Mobile and Ubiquitous Multimedia, MUM 2017
Stuttgart, Germany,

Subject Categories

Interaction Technologies

Human Computer Interaction

Computer Science



More information

Latest update