A New Virtual Keyboard with Finger Gesture Recognition for AR/VR Devices

Abstract

This paper proposes a system that types the virtual keyboard by recognizing hand gestures in a single camera based environment. A virtual keyboard is designed in a multi-tab method of 3 x 4 arrays that are widely used in a mobile environment. In order to make it easy to identify the type-in action and consequently, to increase the type-in speed, this paper proposes a new definition of type-in action as the contact between the thumb and the index finger. To further reduce the input time, the keyboard layout is optimized by efficiently arranging alphabet keys according to the frequency of character appearance. Experimental results show that the proposed type-in action is effective to give commands with a virtual keyboard. Furthermore, the proposed keyboard layout achieves the speed-up by an average of 46.16% compared to the most conventional 'ABC' keyboard.