An Investigation of Composable Language Extensions for Parallel Programming

Abstract

This paper demonstrates how parallel programming language features can be specified as composable language extensions to a general-purpose host programming language. To illustrate the expressiveness of language abstractions defined in this way we have re-implemented, as language extensions, various abstractions previously described in the literature that were implemented as part of traditional monolithic programming languages. These include abstractions for spawning parallel tasks, abstractions implementing bounded buffered channels and lattice-based variables to simplify the communication between parallel tasks, and domain-specific abstractions for simple specification of tensor computations backed by a code-generation technique for the efficient storage and processing of tensor computations. The general-purpose host language is C, as implemented in the AbleC extensible C compiler framework. This system provides certain guarantees of extension composability that ensure that independentlydeveloped language extensions can be automatically composed by programmers that are not experts in language design or implementation. Thus programmers can freely select the abstractions that match their programming problem, their preferred programming style and level of expertise, and their desired performance requirements. This approach also provides benefits to researchers designing and developing new abstractions for parallel programming. It allows them to focus their efforts on the implementation of their new abstractions and re-use the host language implementation of general purpose features such as arithmetic expressions, control-flow statements, type checking, and other basic compiler infrastructure.