Title: The extraction of security situation in heterogeneous log based on Str-FSFDP density peak cluster

Authors: Chundong Wang; Tong Zhao; Xiuliang Mo

Addresses: Key Laboratory of Computer Vision and System, Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Ministry of Education, Tianjin University of Technology, Tianjin, China ' Key Laboratory of Computer Vision and System, Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Ministry of Education, Tianjin University of Technology, Tianjin, China ' Key Laboratory of Computer Vision and System, Tianjin Key Laboratory of Intelligence Computing and Novel Software Technology, Ministry of Education, Tianjin University of Technology, Tianjin, China

Abstract: In order to reduce the false alarm rate in the process of security events extraction and discover a wide range of anomalies by scrutinising various logs, an improvement of Str-FSFDP (a fast search and find of peak density based data stream) clustering algorithm in heterogeneous log analysis is presented. Because of the advantages in data attribute relationship analysis for mixed attributes data, this algorithm can classify log data into two types whose corresponding distance measure metrics are designed. Twelve attributes are defined in the unified XML format for clustering in this paper. These attributes are divided by the characteristics of each type of log and the importance of expressing a security event. To match the new micro cluster characteristic vector mentioned in the Str-FSFDP algorithm, this paper uses time gap to improve the UHAD (unsupervised anomaly detection model) framework. The time gap is designed as a threshold value based on micro cluster strategy. Experimental results reveal that the framework using Str-FSFDP clustering algorithm with time threshold can improve the aggregation rate of the log events and reduce the false alarm rate.

Keywords: heterogeneous log; micro cluster; mixed attributes; unsupervised anomaly detection.

DOI: 10.1504/IJCSE.2019.103943

International Journal of Computational Science and Engineering, 2019 Vol.20 No.3, pp.387 - 396

Received: 19 Nov 2016
Accepted: 03 Jul 2017
Published online: 03 Dec 2019 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article