Title: Bi-directional ConvLSTM-based interactive query focused video summarisation

Authors: Vasudha Tiwari; Charul Bhatnagar

Addresses: Department of CEA, GLA University, Mathura, Uttar Pradesh, India ' Department of CEA, GLA University, Mathura, Uttar Pradesh, India

Abstract: Video summarisation presents the essence of a video in a compact form by extracting the most salient frames from it in a temporal sequence. Although, over time, video summarisation has gained much focus from the researchers, yet the need of personalised summaries that are based on user's intent still needs exploration. The authors propose an interactive, Query Focussed Video Summarisation (QFVS) approach which attempts to find those frames of a video that have maximum pertinence to user's text query. The proposed model consists of a Bi-directional ConvLSTM as an enhancement over ConvLSTM along with Resnet-50 for feature extraction. The input text query is matched with the predicted labels produced by the model and the frames with maximum similarity are selected for summary generation. The proposed method is evaluated on performance with previous state-of-art works and the results clearly demonstrate a significant improvement in the performance and prove the efficiency of the approach.

Keywords: video summarisation; query-based; interactive summarisation; bi-directional ConvLSTM; Resnet-50; text query; LSTM; ConvLSTM.

DOI: 10.1504/IJWMC.2024.137871

International Journal of Wireless and Mobile Computing, 2024 Vol.26 No.3, pp.229 - 237

Received: 25 Jul 2023
Accepted: 27 Dec 2023
Published online: 05 Apr 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article