Proceedings of ISP RAS


Optimization problems running MPI-based HPC applications

Grushin D.A. (ISP RAS, Moscow, Russia)
Kuzjurin N.N. (ISP RAS, Moscow, Russia; MIPT, Dolgoprudny, Moscow Region, Russia)

Abstract

MPI is a well-proven technology that is widely used in a high-performance computing environment. However, configuring an MPI cluster can be a difficult task. Containers are a new approach to virtualization and simple application packaging, which is becoming a popular tool for high-performance tasks (HPC). This approach is considered in this article. Packaging an MPI application as a container solves the problem of conflicting dependencies, simplifies the configuration and management of running applications. A typical queue system (for example, SLURM) or a container management system (Docker Swarm, Kubernetes, Mesos, etc.) can be used to manage cluster resources. Containers also provide more options for flexible management of running applications (stop, restart, pause, in some cases, migration between nodes), which allows you to gain an advantage optimizing the allocation of tasks to cluster nodes in comparison with the classic scheduler. The article discusses various ways to optimize the placement of containers when working with HPC-applications. A variant of launching MPI applications in Fanlight system is proposed, which simplifies the work of users. The optimization problem associated with this method is considered also.

Keywords

docker, containers, scheduling

Edition

Proceedings of the Institute for System Programming, vol. 29, issue 6, 2017, pp. 229-244.

ISSN 2220-6426 (Online), ISSN 2079-8156 (Print).

DOI: 10.15514/ISPRAS-2017-29(6)-14

Full text of the paper in pdf (in Russian) Back to the contents of the volume