What's in the Container? Classifying Object Contents from Vision and Touch
Paper in proceedings, 2014

Robots operating in household environments need to interact with food containers of different types. Whether a container is filled with milk, juice, yogurt or coffee may affect the way robots grasp and manipulate the container. In this paper, we concentrate on the problem of identifying what kind of content is in a container based on tactile and/or visual feedback in combination with grasping. In particular, we investigate the benefits of using unimodal (visual or tactile) or bimodal (visual-tactile) sensory data for this purpose. We direct our study toward cardboard containers with liquid or solid content or being empty. The motivation for using grasping rather than shaking is that we want to investigate the content prior to applying manipulation actions to a container. Our results show that we achieve comparable classification rates with unimodal data and that the visual and tactile data are complimentary.




Robot sensing systems


Puren Guler

Royal Institute of Technology (KTH)

Yasemin Bekiroglu

Chalmers, Signals and Systems, Systems and control, Automatic Control

Xavi Gratal

Royal Institute of Technology (KTH)

Karl Pauwels

Royal Institute of Technology (KTH)

Danica Kragic

Royal Institute of Technology (KTH)

IEEE/RSJ International Conference on Intelligent Robots and Systems

2153-0858 (ISSN) 2153-0866 (eISSN)

IEEE/RSJ International Conference on Intelligent Robots and Systems
Chicago, ,

Subject Categories


Computer Vision and Robotics (Autonomous Systems)

More information


9/2/2020 1