Title: Performance-aware deployment of streaming applications in distributed stream computing systems

Authors: Dawei Sun; Shang Gao; Xunyun Liu; Fengyun Li; Rajkumar Buyya

Addresses: School of Information Engineering, China University of Geosciences, Beijing, 100083, China; Polytechnic Center for Territory Spatial Big-data, MNR of China, China ' School of Information Technology, Deakin University, Victoria 3216, Australia ' Cloud Computing and Distributed Systems (CLOUDS) Laboratory, School of Computing and Information Systems, The University of Melbourne, Australia ' School of Computer Science and Engineering, Northeastern University, Shenyang, 110819, China ' Cloud Computing and Distributed Systems (CLOUDS) Laboratory, School of Computing and Information Systems, The University of Melbourne, Australia

Abstract: Performance-aware deployment of streaming applications is one of the key challenging problems in distributed stream computing systems. We proposed a performance-aware deployment framework (Pa-Stream) for distributed stream computing systems. By addressing the important aspects of the framework, this paper makes the following contributions: 1) investigated the performance-aware deployment of a streaming application over distributed and heterogeneous computing nodes, and provided a general application deployment model; 2) demonstrated a streaming applications deployment scheme by proposing an artificial bee colony strategy that deploys application's vertices onto the best set of computing nodes; an incremental online redeployment strategy was used to redeploy the running application; 3) developed and integrated Pa-Stream into Apache Storm platform; 4) evaluated the fulfilment of low latency and high throughput objectives in a distributed stream computing environment. Experimental results demonstrate that the proposed Pa-Stream provided effective performance improvements on latency, throughput and resource utilisation.

Keywords: performance awareness; application deployment; stream computing; artificial bee colony algorithm; distributed system.

DOI: 10.1504/IJBIC.2020.105892

International Journal of Bio-Inspired Computation, 2020 Vol.15 No.1, pp.52 - 62

Received: 07 Aug 2019
Accepted: 09 Oct 2019
Published online: 16 Mar 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article