A data cleaning method for water quality based on improved hierarchical clustering algorithm Online publication date: Sat, 14-Dec-2019
by Qingxuan Meng; Jianzhuo Yan
International Journal of Simulation and Process Modelling (IJSPM), Vol. 14, No. 5, 2019
Abstract: Identifying and rectifying incomplete water quality data is of vital importance. A data cleaning method based on improved balanced iterative reducing and clustering using hierarchies (BIRCH) clustering algorithm is proposed. The clustering feature tree of water quality data is constructed and the cluster vector of the clustering feature tree is obtained by the agglomerative method. The optimal cluster number is determined according to the Bayesian Information Criterion and the nearest clustering ratio. The Pauta criterion is used to detect the global outlier and artificial neural network (ANN) is used to fill in outliers and missing values. Finally, the improved data cleaning method is applied to water quality monitoring data of Beijing wastewater treatment plant. The experimental results show that the data cleaning method can not only detect abnormal values and missing values accurately, but also normalise and complete missing data.
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Simulation and Process Modelling (IJSPM):
Login with your Inderscience username and password:
Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.
If you still need assistance, please email subs@inderscience.com
Our Blog 
Follow us on Twitter 
Visit us on Facebook