Ways to organize parallel access to structured data

Ways to organize parallel access to structured data

Alexey Olegovich IGNATYEV, Sergey Yurievich MOKSHIN, Dmitry Vladimirovich IVANKOV, Evgeny Alexandrovich BEKETOV

Abstract

This paper explores ways to achieve the highest possible exchange performance with files containing structured data. The research was carried out on file systems with supercomputer systems parallel access designed to solve problems of physical and mathematical modeling of various processes and objects. For example, parallel access to raw data is considered using the Lustre file system. The article suggests a way to organize parallel access to structured data based on a specially developed PSIO storage format and the psio access library.  A comparative analysis of the I/O performance of the developed data storage format and the HDF5 parallel version format is performed.  

Keywords

computing system, structured data, high-performance computing simulation, mathematical simulation subsystem, data format, HDF, HPC, SIO

Edition

Proceedings of the Institute for System Programming, vol. 35, issue 2, 2023, 111-126

ISSN 2220-6426 (Online), ISSN 2079-8156 (Print).

DOI: 10.15514/ISPRAS-2023-35(2)-8

For citation

Alexey Olegovich IGNATYEV, Sergey Yurievich MOKSHIN, Dmitry Vladimirovich IVANKOV, Evgeny Alexandrovich BEKETOV Ways to organize parallel access to structured data. Proceedings of the Institute for System Programming, vol. 35, issue 2, 2023, 111-126 DOI: 10.15514/ISPRAS-2023-35(2)-8.

Full text of the paper in pdf (in Russian) Back to the contents of the volume