Towards higher efficiency in a distributed memory storage system using data compression Online publication date: Thu, 05-Jan-2023
by Xiaoyang Yu; Songfeng Lu; Tongyang Wang; Xinfang Zhang; Shaohua Wan
International Journal of Bio-Inspired Computation (IJBIC), Vol. 20, No. 4, 2022
Abstract: As the amount of data grows, achieving an appropriate trade-off among computation, storage and network transportation will be beneficial for a distributed memory storage system, leading to higher overall efficiency. To this end, we explore a method to achieve this trade-off by introducing data compression technology in a transparent manner. Instead of focusing on specific compressed data structures, we target block level compression for a general-purpose storage system to incorporate a wide range of existing data analysis frameworks and usage scenarios, especially with big data. A prototype is implemented and evaluated based on the memory-centric distributed storage system Alluxio to provide transparent compression and decompression during write/read operations. The extensive experiments for data with different types of compression ratio are conducted and the experimental results prove that our approach can achieve huge write/read throughput.
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Bio-Inspired Computation (IJBIC):
Login with your Inderscience username and password:
Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.
If you still need assistance, please email subs@inderscience.com
Our Blog 
Follow us on Twitter 
Visit us on Facebook