Article: A scalable Map Reduce tasks scheduling: a threading-based approach Journal: International Journal of Computational Science and Engineering (IJCSE) 2017 Vol.14 No.1 pp.44 - 54 Abstract: The Map Reduce paradigm is now considered a standard platform that is used for large-scale data processing and management. A major operation that the Map Reduce platform relies on greatly is tasks scheduling. Although many schedulers have been presented, task scheduling is still one of the major problems that face Map Reduce frameworks. Schedulers need to maintain data locality to achieve an acceptable performance by avoiding several data transmissions. Hence, in this paper, we propose a new scheduling algorithm named 'MTL' that utilises multi-threading principles. The MTL scheduler assigns a dedicated thread for each data block. Indeed, the multi-threading approach shows great results that make our MTL scheduler a scalable one that performs well. At the same time, it maintains the locality property. During the evaluation of the MTL scheduler performance, two main factors were taken into consideration; the simulation time and the energy consumption. The MTL scheduler is then compared with other existing schedulers such as FIFO, matchmaking, and delay schedulers. The MTL scheduler showed favourable results and proved its advantages over other existing schedulers. Inderscience Publishers - linking academia, business and industry through research

Title: A scalable Map Reduce tasks scheduling: a threading-based approach

Authors: Qutaibah Althebyan; Omar AlQudah; Yaser Jararweh; Qussai Yaseen

Addresses: Software Engineering Department, Jordan University of Science and Technology, Irbid, 22110, Jordan ' Computer Science Department, Jordan University of Science and Technology, Irbid, 22110, Jordan ' Computer Science Department, Jordan University of Science and Technology, Irbid, 22110, Jordan ' Computer Information Systems Department, Jordan University of Science and Technology, Irbid, 22110, Jordan

Abstract: The Map Reduce paradigm is now considered a standard platform that is used for large-scale data processing and management. A major operation that the Map Reduce platform relies on greatly is tasks scheduling. Although many schedulers have been presented, task scheduling is still one of the major problems that face Map Reduce frameworks. Schedulers need to maintain data locality to achieve an acceptable performance by avoiding several data transmissions. Hence, in this paper, we propose a new scheduling algorithm named 'MTL' that utilises multi-threading principles. The MTL scheduler assigns a dedicated thread for each data block. Indeed, the multi-threading approach shows great results that make our MTL scheduler a scalable one that performs well. At the same time, it maintains the locality property. During the evaluation of the MTL scheduler performance, two main factors were taken into consideration; the simulation time and the energy consumption. The MTL scheduler is then compared with other existing schedulers such as FIFO, matchmaking, and delay schedulers. The MTL scheduler showed favourable results and proved its advantages over other existing schedulers.

Keywords: MapReduce tasks; task scheduling; multi-threading; clustering; Hadoop; scalability; threading; simulation; energy consumption.

DOI: 10.1504/IJCSE.2017.081175

International Journal of Computational Science and Engineering, 2017 Vol.14 No.1, pp.44 - 54

Received: 29 May 2014
Accepted: 06 Nov 2014
Published online: 26 Dec 2016 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: A scalable Map Reduce tasks scheduling: a threading-based approach

Keep up-to-date