Elastic Job Bundling: An Adaptive Resource Request Strategy for Large-Scale Parallel Applications

Liu, Feng
Weissman, Jon
2015-04-16
Thumbnail Image

View/Download File

15-006.pdf (662.94 KB)

Persistent link to this item

https://hdl.handle.net/11299/215971
Statistics
View Statistics

Journal Title

Journal ISSN

Volume Title

Title

Elastic Job Bundling: An Adaptive Resource Request Strategy for Large-Scale Parallel Applications

Authors

Published Date

2015-04-16

Publisher

Type

Report

Abstract

In today’s batch queue HPC cluster systems, the user submits a job requesting a fixed number of processors. The system will not start the job until all of the requested resources become available simultaneously. When cluster workload is high, large sized jobs will experience long waiting time due to this policy. In this paper, we propose a new approach that dynamically decomposes a large job into smaller ones to reduce waiting time, and lets the application expand across multiple subjobs while continuously achieving progress. This approach has three bene?ts: (i) application turnaround time is reduced, (ii) system fragmentation is diminished, and (iii) fairness is promoted. Our approach does not depend on job queue time prediction but exploits available back?ll opportunities. Simulation results have shown that our approach can reduce application mean turnaround time by up to 48%.

Keywords

Description

Related to

Replaces

License

Collections

Computer Science & Engineering (CS&E) Technical Reports

Series/Report Number

Technical Report; 15-006

Funding information

Isbn identifier

Doi identifier

Previously Published Citation

Suggested citation

Liu, Feng; Weissman, Jon. (2015). Elastic Job Bundling: An Adaptive Resource Request Strategy for Large-Scale Parallel Applications. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/215971.

Content distributed via the University Digital Conservancy may be subject to additional license and use restrictions applied by the depositor. By using these files, users agree to the Terms of Use. Materials in the UDC may contain content that is disturbing and/or harmful. For more information, please see our statement on harmful content in digital repositories.