Challenges in Deictic Gesture-Based Spatial Referencing for Human-Robot Interaction in Construction

Abstract

As robots are envisioned to be deployed in construction job sites to work with humans, there is an increasing need for developing intuitive and natural communication between robots and humans. In particular, spatial information exchange is critical to navigating or delegating tasks to collaborative robots. However, such deictic gestures are inherently imprecise and ambiguous. Thus, it is challenging for robots to reason about the exact region of interest, especially in a cluttered large-scale construction environment. To address this limitation, this study evaluates the performance of spatial information exchange through the experiments based on pointing targets on the wall and ceiling, which are the most common workspaces in construction. We observed that the current deictic gesture-based method can estimate the pointed position on the wall and ceiling with a mean distance error of 0.767m, while the error tends to increase by 0.715m in the ceiling and 0.115m in the side panels. Our experimental results indicate that the deictic gesture-based method has some challenges in ceiling and side panel conditions, while the overall panel recognition shows acceptable performance. The findings of this study will help novice construction workers naturally and effectively communicate with robots by delivering spatial information on specific objects or regions in the shared workspace.