Operation of an Intelligent Space with Heterogeneous Sensors

Abstract

A new approach of multi-sensor operation is presented in an intelligent space, which is based on heterogeneous multiple vision sensors and robots mounted with an infrared (IR) sensor. The intelligent space system is a system that exists in task space of robots, helps missions of the robots, and can self-control the robots in a particular situation. The conventional intelligent space consists of solely static cameras. However, the adoption of multiple heterogeneous sensors and an operation technique for the sensors are required in order to extend the ability of intelligent space. First, this dissertation presents the sub-systems for each sensor group in the proposed intelligent space. The vision sensors consist of two groups: static (fixed) cameras and dynamic (pan-tilt) cameras. Each sub-system can detect and track the robots. The sub-system using static cameras localize the robot within a high degree of accuracy. In this system, a handoff method is proposed using the world-to-pixel transformation in order to interwork among the multiple static cameras. The sub-system using dynamic cameras is designed to have various views without losing the robot in view. In this system, a handoff method is proposed using the predictive positions of the robot, relationship among cameras, and relationship between the robot and the camera in order to interwork among the multiple dynamic cameras. The robots system localizes itself using an IR sensor and IR tags. The IR sensor can localize the robot even if illumination of the environment is low. For robust tracking, a sensor selection method is proposed using the advantages of these sensors under environmental change of the task space. For the selection method, we define interface protocol among the sub-systems, sensor priority, and selection criteria. The proposed method is adequate for a real-time system, which has a low computational cost than sensor fusion methods. Performance of each sensor group is verified through various experiments. In addition, multi-sensor operation using the proposed sensor selection method is experimentally verified in the environment with an occlusion and low-illumination setting.