Saliency Detection Methods Based on Multi-seed Propagation on Multilayer Graphs

Abstract

Humans are able to understand visual information very quickly, even in complex scenes by unintentionally prioritizing the regions or objects in the scenes and spending their efforts to interpret things according to their priorities. Visual saliency detection is to find such important and noticeable things in images as humans do. It is exploited as a preprocessing tool of a variety of tasks in computer vision and image processing area. Thus, there have been many saliency detection methods which have achieved to locate salient objects in a certain degree. Recently, many researchers have focused on uniformly highlighting the salient objects via diverse approaches. In this respect, this dissertation presents novel methods for multilayer graph construction and multi-seed propagation to detect salient objects more accurately, which is applied to three topics: single image saliency detection, co-saliency detection of multiple images, and skin detection as a target specific task.

First, a single image saliency detection method is proposed based on the seed propagation approach on a graph. Unlike existing approaches, the proposed method exploits two different types of seeds for the propagation of saliency information, each of which stands for salient objects and background respectively. Two kinds of seeds are separately propagated to all nodes of a graph which is effectively learned by a semi-supervised learning scheme, and the saliency map is generated by combining the results of each propagation. In addition, this approach is expanded to multilayer graphs for better localization of salient objects homogeneously. Two different methods are presented in this dissertation, one of which maintains spatial coherence and another method focuses on increasing the consistency on feature space, e.g., color feature. Hence, both multilayer graph cases take advantage over the single layer graph case in a such way that these consider global and local relations of a whole image. Experiments demonstrate that introducing the multi seeds helps to reduce false positives and the performance is further improved by constructing the multilayer graphs. Moreover, the proposed approach outperforms the state-the-art methods in terms of various objective measures.

This dissertation also presents a method for co-saliency detection which aims at locating salient objects occurring in multiple images. Unlike saliency detection, it needs additional information to represent coherence of regions among images, so a pairwise coherence cue is designed to describe co-existence as well as saliency from similarity of saliency (SoS). The basic framework of the proposed method follows that of the proposed single saliency detection approach which mainly consists of multi seed extraction, graph construction and propagation steps. However, it is necessary that the graphs of each image are connected in order to take account of the saliency information of other images together and to propagate the seeds over the nodes of every image. The nodes between different images are indirectly connected via additional nodes (cluster nodes), because it is a very challenging problem to link those inter nodes directly when the scales and locations of salient objects are different among images in general. Experiments demonstrate that the co-saliency cues of each image are successfully transferred to all image nodes, and it is also shown that the proposed co-saliency detection method yields better results on objective measures and visually plausible results compared to the state-of-the-art methods.

Lastly, the approach of multi-seed propagation on a multilayer graph is utilized in a specific classification task, i.e., skin detection which extracts skin pixels/regions in images, where the regions are mostly considered as salient when there exist humans in the images. Instead of saliency seeds, skin seeds are exploited along with background seeds for the propagation step, which can be achieved by adopting existing skin detectors. The graph can be connected more relevantly with skin information, because it is possible to use top-down features unlike most saliency detection methods. Experiments shows that the proposed method outperforms other methods including the classical skin detection method used for the seed extraction, which demonstrates that the proposed approach can be used for the skin detection, and it is expected that the proposed framework may also be utilized in other image segmentation/classification problems.