A HDFS dynamic load balancing strategy using improved niche PSO algorithm in cloud storage Online publication date: Thu, 15-Apr-2021
by Zhiyu Jian; Yiwei Jian
International Journal of Autonomous and Adaptive Communications Systems (IJAACS), Vol. 14, No. 1/2, 2021
Abstract: A Hadoop distributed file system (HDFS) NameNode dynamic load balancing strategy (NDLBT) using improved niche particle swarm optimisation (PSO) is proposed, which is used to solve the load balancing problem of urban surveillance video big data. Firstly, the relationship between bandwidth consumption and the bit rate of video files, the size of data blocks and access hotspots of online video files are analysed. Then, adaptive backup is realised by dynamic multiple copies of heterogeneous nodes. The dynamic distribution of metadata is implemented to ensure the performance of metadata server clusters. Finally, we propose a new improved niche PSO algorithm to achieve load balancing scheduling. In the experimental scenario where high bandwidth consumption video files are used as service access hotspots, the proposed method, which can reduce the bandwidth peak value of bottleneck nodes in data node clusters by 20%, is superior to the original load balancing method in 90% scenarios.
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Autonomous and Adaptive Communications Systems (IJAACS):
Login with your Inderscience username and password:
Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.
If you still need assistance, please email subs@inderscience.com
Our Blog 
Follow us on Twitter 
Visit us on Facebook