Title: Vocabulary hierarchy optimisation based on spatial context and category information

Authors: Zhiguo Yang; Yuxin Peng; Jianguo Xiao

Addresses: Institute of Computer Science and Technology, Peking University, Beijing, 100871, China ' Institute of Computer Science and Technology, Peking University, Beijing, 100871, China ' Institute of Computer Science and Technology, Peking University, Beijing, 100871, China

Abstract: In this paper, we focus on the hierarchy and discriminating ability of visual vocabulary. We propose to use the category information of images and the spatial context of keypoints to select appropriate visual words from different hierarchical levels. Existing approaches, such as flat vocabulary and vocabulary tree, can change the hierarchy of all visual words at the same time, by setting different cluster numbers and tree height respectively. However, the most appropriate visual words may be at different hierarchical levels, and existing approaches could not adjust the hierarchy of different visual words separately. To address this problem, we propose an object function to describe the consistence of visual words, with category information of images and spatial context of keypoints, and then we adopt simulated annealing algorithm to search for a sub-optimal solution, which corresponds to a visual vocabulary selected from the vocabulary tree. Different from existing methods, the proposed approach can select the most appropriate visual words from different levels adaptively, which can improve the performances in image annotation and classification tasks. Experiments on widely-used 15-scenes dataset demonstrate the effectiveness of the proposed approach.

Keywords: bag-of-visual-words; BoVW; vocabulary tree; spatial context; category information; hierarchy selection; simulated annealing; vocabulary hierarchy optimisation; visual vocabulary; visual words; image classification; image annotation.

DOI: 10.1504/IJMIS.2013.056470

International Journal of Multimedia Intelligence and Security, 2013 Vol.3 No.1, pp.93 - 107

Published online: 26 Jul 2014 *