Title: Detection of stragglers and optimal rescheduling of slow running tasks in big data environment using LFCSO-LVQ classifier and enhanced PSO algorithm

Authors: Hetal A. Joshiara; Chirag S. Thaker; Sanjay M. Shah; Darshan B. Choksi

Addresses: Department of Computer Engineering, Gujarat Technological University, Chandkheda, Gujarat, India ' Department of Computer Engineering, Gujarat Technological University, Chandkheda, Gujarat, India ' Department of Computer Engineering, Gujarat Technological University, Chandkheda, Gujarat, India ' Department of Computer Engineering, Sardar Patel University, Chandkheda, Gujarat, India

Abstract: This paper plans to implement intelligent techniques in finding straggler tasks along with speculating their way of execution. Here, the LFCSO-LVQ is proposed to effectively identify the slow running tasks as of a bunch of user tasks, and the enhanced PSO is proposed for performing optimal rescheduling of the identified SR tasks. Initially, the collected data are preprocessed by means of identifying homogenous and heterogeneous tasks. After that, the Apache Spark split the preprocessed tasks into several sub-tasks. The features are extracted as of these subtasks for SR task prediction. An information gain-based linear discriminant analysis is proposed for feature selection approach that reduces the classifier's training time. Subsequent to FS, the selected ones are inputted to LFCSO-LVQ, which envisages the SR tasks of the dataset centred on the chosen features. After that, EPSO reschedules these predicted tasks to the other fastest nodes of virtual machine.

Keywords: big data; straggler detection; rescheduling of tasks; speculative execution; slow running tasks identification; learning vector quantisation; LVQ; optimal resource scheduling; particle swarm optimisation; PSO.

DOI: 10.1504/IJDATS.2022.121505

International Journal of Data Analysis Techniques and Strategies, 2022 Vol.14 No.1, pp.1 - 21

Received: 16 Jan 2021
Accepted: 23 Apr 2021
Published online: 16 Mar 2022 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article