Evaluating the usability of sign language gestures for gesture-based human-machine interaction

Abstract

Utilizing gestures of major sign languages (signs) for gesture-based interaction seems to be an appealing idea as it has some obvious advantages, including: reduced time and cost for gesture vocabulary design, immediate accommodation of existing sign language users and supporting universal design and equality by design. However, it is not well understood whether or not sign language gestures are indeed adequate for gesture-based interaction, especially in terms of usability. As an initial effort to enhance our understanding of the usability of sign language gestures, the current study evaluated Korean Sign Language (KSL) gestures employing three usability criteria: intuitiveness, preference and physical stress. A set of 18 commands for manipulating objects in virtual worlds was determined. Then, gestures for the commands were designed using two design methods: the sign language method and the user design method. The sign language method consisted of simply identifying the KSL gestures corresponding to the commands. The user design method involved having user representatives freely design gestures for the commands. A group of evaluators evaluated the resulting sign language and user-designed gestures in intuitiveness and preference through subjective ratings. Physical stresses of the gestures were quantified using an index developed based on Rapid Upper Limb Assessment. The usability scores of the KSL gestures were compared with those of the user-designed gestures for relative evaluation. Data analyses indicated that overall, the use of the KSL gestures cannot be regarded as an excellent design strategy when viewed strictly from a usability standpoint, and the user-design approach would likely produce more usable gestures than the sign language approach if design optimization is performed using a large set of user-designed gestures. Based on the study findings, some gesture vocabulary design strategies utilizing sign language gestures are discussed. The study findings may inform future gesture vocabulary design efforts.