SRAM: Saliency aided Recurrent Attention Model for Image Classification

Abstract

To overcome the poor scalability of convolutional neural network, recurrent attention model(RAM) selectively choose what and where to look on the image. By directing recurrent attention model how to look the image, RAM can be even more successful in that the given clue narrows down the scope of the possible focus zone. In this per- spective, this work proposes a novel model, saliency aided recurrent attention model (SRAM) which adopts visual saliency as additional information for image classifi- cation. First, a saliency detector with an intuitive structure for extracting image saliency information was constructed. By combining saliency map as an additional image channel, visual saliency presented as a milestone for viewing the image. SRAM follows encoder-decoder framework, encoder utilizes recurrent attention model with spatial transformer network and decoder for classification. To ensure the perfor- mance, we evaluate the SRAM on the cifar10 dataset and showing 2.209% decrease at most in error rate compared to RAM model.