Comparative Analysis of Container for High Performance Computing (고성능 컴퓨팅을 위한 컨테이너 기술 비교 분석 연구)

Lee Jaeryun (이재륜); Yunchang Chae (채윤창); Byungchul Tak (탁병철)

doi:10.9708/jksci.2020.25.09.011

Comparative Analysis of Container for High Performance Computing

Journal of The Korea Society of Computer and Information
Abbr : JKSCI
2020, 25(9), pp.11-20
DOI : 10.9708/jksci.2020.25.09.011
Publisher : The Korean Society Of Computer And Information
Research Area : Engineering > Computer Science
Received : August 3, 2020
Accepted : September 11, 2020
Published : September 29, 2020

Lee Jaeryun ¹, Yunchang Chae ¹, Byungchul Tak ¹

¹경북대학교

Accredited

ABSTRACT

In this paper, we propose the possibility of using containers in the HPC ecosystem and the criteria for selecting a proper PMI library. Although demand for container has been growing rapidly in the HPC ecosystem, Docker container which is the most widely used has a potential security problem and is not suitable for the HPC. Therefore, several HPC containers have appeared to solve this problem and the chance of performance differences also emerged. For this reason, we measured the performance difference between each HPC container and Docker container through NAS Parallel Benchmark experiment and checked the effect of the type of PMI library. As a result, the HPC container and the Docker container showed almost the same performance as native, or in some cases, rather better performance was observed. In the result of comparison between PMI libraries showed that PMIx was not superior to PMI-2 in all conditions.

KEYWORDS

Container, HPC, PMI, Singularity, Shifter, Charliecloud

Citation status

* References for papers published after 2023 are currently being built.

[confproc] D.M. Jacobsen / 2015 / Contain this, unleashing docker for hpc / Proceedings of the Cray User Group : 33~49

[journal] Gregory M. Kurtzer / 2017 / Singularity: Scientific containers for mobility of compute / PLOS ONE / Public Library of Science (PLoS) 12(5) : e0177459~

[confproc] R. Priedhorsky / 2017 / Charliecloud: unprivileged containers for user-defined software stacks in HPC / Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2017 : 1~10

[journal] L. Gerhardt / 2017 / Shifter : Containers for HPC / Journal of Physics: Conference Series 898(8) : 082021~

[confproc] P. Balaji / 2010 / PMI: A scalable parallel process-management interface for extreme-scale systems / European MPI Users' Group Meeting : 31~41

[journal] Ralph H. Castain / 2018 / PMIx: Process management for exascale environments / Parallel Computing / Elsevier BV 79 : 9~29

[confproc] S. Chakraborty / 2016 / SHMEMPMI -- Shared Memory Based PMI for Improved Performance and Scalability / 2016 16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) : 60~69

[journal] D. H. Bailey / 1991 / The nas parallel benchmarks / International Journal of High Performance Computing Applications 5(3) : 63~73

[journal] H. Pötzl / 2007 / Container-based operating system virtualization: A scalable, high-performance alternative to hypervisors / ACM SIGOPS Operating Systems Review : 275~287

[confproc] M. G. Xavier / 2013 / Performance Evaluation of Container-Based Virtualization for High Performance Computing Environments / 2013 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing : 233~240

[confproc] W. Felter / 2015 / An updated performance comparison of virtual machines and linux containers / 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) : 171~172

[confproc] M. de Bayser / 2017 / Integrating MPI with Docker for HPC / 2017 IEEE International Conference on Cloud Engineering (IC2E) : 259~265

[confproc] A. Azab / 2017 / Enabling Docker Containers for High-Performance and Many-Task Computing / 2017 IEEE International Conference on Cloud Engineering (IC2E) : 279~285

[journal] Jonathan Sparks / 2018 / Enabling Docker for HPC / Concurrency and Computation: Practice and Experience / Wiley 31(16) : e5018~

[confproc] A. J. Younge / 2017 / A Tale of Two Systems Using Containers to Deploy HPC Applications on Supercomputers and Clouds / 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom) : 74~81

[confproc] O. Rudyy / 2019 / Containers in hpc: A scalability and portability study in production biological simulations / 2019IEEE International Parallel and Distributed Processing Symposium (IPDPS)

[confproc] A. Torrez / 2019 / HPC container runtimes have minimal or no performance impact / 2019IEEE/ACM International Workshop on Containers and New Orchestration Paradigms for Isolated Environments in HPC (CANOPIE-HPC) : 37~42

This paper was written with support from the National Research Foundation of Korea.

KJCKorea
Journal Central

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

Comparative Analysis of Container for High Performance Computing

ABSTRACT

KEYWORDS

Citation status

* References for papers published after 2023 are currently being built.

Journal of The Korea Society of Computer and Information 2023 KCI Impact Factor : 0.65

Comparative Analysis of Container for High Performance Computing

ABSTRACT

KEYWORDS

Statistics

Tools

Issue List

Citation status

KCI Citation Counts (0)

REFERENCES (17) * References for papers published after 2023 are currently being built.

Search PDF

Citation

* References for papers published after 2023 are currently being built.