Title: Cross-media retrieval: state-of-the-art and open issues

Authors: Jing Liu, Changsheng Xu, Hanqing Lu

Addresses: Institute of Automation, Chinese Academy of Sciences, 95 Zhongguancun East Road, Beijing 100190, China; China-Singapore Institute of Digital Media, 21 Heng Mui Keng Terrace, Singapore 119613, Singapore. ' Institute of Automation, Chinese Academy of Sciences, 95 Zhongguancun East Road, Beijing 100190, China; China-Singapore Institute of Digital Media, 21 Heng Mui Keng Terrace, Singapore 119613, Singapore. ' Institute of Automation, Chinese Academy of Sciences, 95 Zhongguancun East Road, Beijing 100190, China; China-Singapore Institute of Digital Media, 21 Heng Mui Keng Terrace, Singapore 119613, Singapore

Abstract: Cross-media retrieval is able to provide retrieval results with similar semantics but different media to the query. Since cross-media retrieval complements currently popular text/content-based retrieval which provides retrieval results with the same media to the query, it becomes an emerging research topic. In this paper, we give an overview of recent advances and research challenges in cross-media retrieval. Such retrieval paradigm has been driven by the wide availability of large multimedia resources on the web, innovative approaches to semantically understanding various multimedia objects, and novel machine learning techniques for the interactive mappings among heterogeneous feature spaces. Emphasising on these respects, we review the state-of-the-art approaches and achieved progresses in cross-media retrieval. According to the current technologies used in cross-media retrieval and the demand from real-world applications, open research issues and future research opportunities are identified.

Keywords: cross-media retrieval; semantic understanding; multimedia correlation; machine learning; multimodality; semantics; multimedia content retrieval.

DOI: 10.1504/IJMIS.2010.035970

International Journal of Multimedia Intelligence and Security, 2010 Vol.1 No.1, pp.33 - 52

Published online: 11 Oct 2010 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article