Article: Application-bypass reduction for large-scale clusters Journal: International Journal of High Performance Computing and Networking (IJHPCN) 2004 Vol.2 No.2/3/4 pp.99 - 109 Abstract: Process skew is an important factor in the performance of parallel applications, especially in large-scale clusters. Reduction is a common collective operation which, by its nature, introduces implicit synchronisation between the processes involved in the communication and is therefore highly susceptible to performance degradation due to process skew. A collective operation with application-bypass does not require the application to block in order for the operation to make progress. Application-bypass collective operations are therefore highly tolerant of skew. In this paper, we describe the design and implementation of an application-bypass version of the reduction operation in MPICH over GM. We evaluate our implementation on a 32-node cluster. Under conditions of process skew we find a factor of improvement of up to 5.1 for our application-bypass reduction versus the default MPICH implementation. In addition, we see that this factor of improvement increases with system size, indicating that the application-bypass implementation is more scalable and skew-tolerant than the default non-application-bypass version. This framework promises design and development of high-performance and scalable collective communication libraries for next-generation large-scale clusters. Inderscience Publishers - linking academia, business and industry through research

Title: Application-bypass reduction for large-scale clusters

Authors: Adam Wagner, Darius Buntinas, Ron Brightwell, Dhabaleswar K. Panda

Addresses: Northrop Grumman Xetron, 460W. Crescentville Road, Cincinnati, OH 43210, USA. ' Mathematics and Computer Science Division, Argonne National Laboratory, 9700 South Cass Avenue, Argonne, IL 60439, USA. ' Scalable Computing System Department, Sandia National Laboratories, Albuquerque, MN 87185, USA. ' Department of Computer Science and Engineering, The Ohio State University, Columbus, OH 43120, USA

Abstract: Process skew is an important factor in the performance of parallel applications, especially in large-scale clusters. Reduction is a common collective operation which, by its nature, introduces implicit synchronisation between the processes involved in the communication and is therefore highly susceptible to performance degradation due to process skew. A collective operation with application-bypass does not require the application to block in order for the operation to make progress. Application-bypass collective operations are therefore highly tolerant of skew. In this paper, we describe the design and implementation of an application-bypass version of the reduction operation in MPICH over GM. We evaluate our implementation on a 32-node cluster. Under conditions of process skew we find a factor of improvement of up to 5.1 for our application-bypass reduction versus the default MPICH implementation. In addition, we see that this factor of improvement increases with system size, indicating that the application-bypass implementation is more scalable and skew-tolerant than the default non-application-bypass version. This framework promises design and development of high-performance and scalable collective communication libraries for next-generation large-scale clusters.

Keywords: application-bypass; reduction; collective communications; process skew; heterogeneous; cluster computing; MPI; MPICH; GM; Myrinet.

DOI: 10.1504/IJHPCN.2004.008896

International Journal of High Performance Computing and Networking, 2004 Vol.2 No.2/3/4, pp.99 - 109

Published online: 02 Feb 2006 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: Application-bypass reduction for large-scale clusters

Keep up-to-date