Title: A multi-modal image encoding and self-attention-based transformer framework with sentiment analysis for financial time series prediction

Authors: Ravi Prakash Varshney; Dilip Kumar Sharma

Addresses: Department of Computer Engineering and Applications, GLA University, Mathura, India ' Department of Computer Engineering and Applications, GLA University, Mathura, India

Abstract: In this paper, we propose a novel approach for financial time series forecasting using feature selection, image encoding, and a self-attention-based CNN transformer. We use Markov transition field and candlestick chart encoding to extract features from historical stock data. Additionally, we incorporate the sentiment analysis of the financial news data in our model to improve the forecast accuracy. The proposed approach is compared to traditional time series forecasting methods, and the results show that our method outperforms the traditional method in terms of forecasting accuracy. The proposed approach can be used to improve risk management and make more informed trading decisions. Our experiments demonstrate that the proposed framework achieved an improvement of approximately 17.8% in root mean squared error and ~38.7% in mean absolute error for securities lending dataset and ~71.5% improvement in root mean squared error and around ~83.2% improvement in mean absolute error for pricing dataset.

Keywords: candlestick image encoding; computer vision; convolutional neural network transformer; feature selection; Markov transition field image encoding; multivariate time series; MTS; pattern recognition; long-short term memory; LSTM; sentimental analysis.

DOI: 10.1504/IJCVR.2025.142920

International Journal of Computational Vision and Robotics, 2025 Vol.15 No.1, pp.31 - 58

Received: 28 Jan 2023
Accepted: 03 May 2023
Published online: 02 Dec 2024 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article