Role of Message-Passing in Performance Oriented Parallel Programming

Abstract

The message-passing programming paradigm is often called the assembly language of parallel computers. The paradigm requires the user to partition the data among processors and specify interaction among processors explicitly. In contrast, the shared memory programming paradigm offers the user a global address space, making it unnecessary to specify data partitioning and communication. Indeed, many parallel programs written using the shared-memory paradigm are much shorter than their counterparts in the message-passing paradigm. Until recently, cache-coherent shared address space architectures were largely based on a global shared bus, making them inherently unsealable beyond a few dozen processors. More recently, scalable directory-based cache-coherent shared address space architecture are being offered by many vendors. This seems to make the message-passing programming style obsolete. In this article, we argue that the style of message-passing programming makes it much easier to develop efficient parallel programs even on cache-coherent shared address space architectures.