Title: A cloud/edge computing streaming system for network traffic monitoring and threat detection

Authors: Zhijiang Chen; Sixiao Wei; Wei Yu; James H. Nguyen; William Grant Hatcher

Addresses: Intelligent Fusion Technology, Inc., 20271 Goldenrod Ln, Germantown, MD 20876, USA ' Intelligent Fusion Technology, Inc., 20271 Goldenrod Ln, Germantown, MD 20876, USA ' Department of Computer and Information Science, Towson University, Towson, MD, USA ' Department of Computer and Information Science, Towson University, Towson, MD, USA ' Department of Computer and Information Science, Towson University, Towson, MD, USA

Abstract: The unyielding trend of increasing cyber threats has made cyber security paramount in protecting personal and private intellectual property. To provide a highly secured network environment, network threat detection systems must handle real-time big data from varied places in enterprise networks. In this paper, we introduce a streaming-based threat detection system that can rapidly analyse highly intensive network traffic data in real-time, utilising streaming-based clustering algorithms to detect abnormal network activities. The developed system integrates the high-performance data analysis capabilities of Flume, Spark and Hadoop into a cloud-computing environment to provide network monitoring and intrusion detection. Our performance evaluation validates that the developed system can cope with a significant volume of streaming data in a high detection accuracy and good system performance. We further extend our system for edge computing and discuss some key challenges, as well as some potential solutions, aiming to improve the scalability of our system.

Keywords: streaming analysis; network traffic monitoring; threat detection; big network data analysis; cloud computing; edge computing.

DOI: 10.1504/IJSN.2018.093551

International Journal of Security and Networks, 2018 Vol.13 No.3, pp.169 - 186

Received: 24 Oct 2017
Accepted: 30 Nov 2017
Published online: 27 Jul 2018 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article