Adaptive Parallelism for Scientific Applications

Adaptive Parallelism for Scientific Applications

Adaptive Parallelism for Scientific Applications
----------------------------------------------------------------------------
Scientific applications demand large computation and have data parallelism by nature. Parallel scientific applications are developed to leverage supercomputers and cloud infrastructures. The ideal amount of computing resources to use varies with queries workload and computing infrastructures.Furthermore, users may have limited budget or time constraints.This paper presents APlug, an adaptive parallelism framework for scientific applications. APlug estimates the degree of parallelism and dynamically recommends the number of cores considering workload per query, user constraints, and infrastructure. We utilize a Gamma probability density function to eliminate estimation errors due to extreme skewness inherent in the set of tasks. Our experiments show the viability of our framework for molecular docking, sequence alignment, and string mining under large-scale infrastructures; 16,384 CPUs in a supercomputer and 480 cores in a Linux cluster. APlug estimates the degree of parallelism in very short time with less than 10% error.​
​
Download