A framework for collective I/O style optimisations at staging I/O nodes
by Vishwanath Venkatesan; Mohamad Chaarawi; Quincey Koziol; Neil Fortner; Edgar Gabriel
International Journal of Big Data Intelligence (IJBDI), Vol. 3, No. 2, 2016

Abstract: Data-intensive applications are largely influenced by the I/O performance on high performance computing systems, and the scalability of such applications primarily depends on the scalability of the I/O subsystem itself. To mitigate the effects of I/O operations, recent high end systems make use of staging nodes to delegate I/O requests and thus decouple I/O and compute operations. The data staging layers however, lack the benefit that could be obtained from the collective I/O style optimisations that client side parallel I/O operations provide. In this paper, we present the compactor framework developed as a part of the exascale fastforward I/O stack. The compactor framework introduces optimisations at the data staging nodes, including collective buffering across requests from multiple processes, write stealing to service read requests at the staging node, and write morphing to optimise multiple write requests from the same process. The compactor framework is evaluated on a PVFS2 file system using micro-benchmarks, the flash I/O benchmark and a parallel image processing application. Our results indicate significant performance benefits of up to 70% due to the optimisations of the compactor.

Online publication date: Wed, 29-Jun-2016

The full text of this article is only available to individual subscribers or to users at subscribing institutions.

 
Existing subscribers:
Go to Inderscience Online Journals to access the Full Text of this article.

Pay per view:
If you are not a subscriber and you just want to read the full contents of this article, buy online access here.

Complimentary Subscribers, Editors or Members of the Editorial Board of the International Journal of Big Data Intelligence (IJBDI):
Login with your Inderscience username and password:

Username:        Password:          Forgotten your password?

Want to subscribe?
A subscription gives you complete access to all articles in the current issue, as well as to all articles in the previous three years (where applicable). See our Orders page to subscribe.

If you still need assistance, please email subs@inderscience.com