Proceedings of ISP RAS


Implementing Apache Spark jobs execution and Apache Spark cluster creation for Openstack Sahara

A. Aleksiyants (ISP RAS, Moscow), O. Borisenko (ISP RAS, Moscow), D. Turdakov (ISP RAS, Moscow; MSU, Moscow; HSE, Moscow), A. Sher (ISP RAS, Moscow), S. Kuznetsov (ISP RAS, Moscow; MSU, Moscow; MIPT, Moscow)

Abstract

In this paper the problem of creating virtual clusters in clouds for big data analysis with Apache Hadoop and Apache Spark is discussed. Existing methods for Apache Spark clusters creation are described in this work. Also the implemented solution for building Apache Spark clusters and Apache Spark jobs execution in Openstack environment is described. The implemented solution is a modification for OpenStack Sahara project and it was featured in Openstack Liberty release.

Keywords

Apache Spark, Openstack, Openstack Sahara, IaaS, PaaS

Edition

Proceedings of the Institute for System Programming, vol. 27, issue 5, 2015, pp. 35-48.

ISSN 2220-6426 (Online), ISSN 2079-8156 (Print).

DOI: 10.15514/ISPRAS-2015-27(5)-3

Full text of the paper in pdf Back to the contents of the volume