Home / Apche Spark

How to Install Spark on Google Compute Engine

Rating: 4.0Blog-star
Views: 2301
by Ravindra Savaram
Last modified: May 7th 2020

Spark On Google Compute Engine

GCE stands for Google Compute Engine. It is an infrastructure as a service which lets you execute your large scale computing workloads on the virtual machines which have been hosted at the infrastructure for Google. It is also possible for you to create your own and new machine and this will take you a very short time.

Are you intereted in taking up for Apache Spark Certification Training? Enroll for Free Demo on  Apache Spark Training!

The following are the installation steps:

1. Begin by creating a CentOS image on the GCE. The space used should be at least 3.8 GB, otherwise the system will not compile.

2. Once you are done, you can perform an ssh into the new machine. An example of how this can be done is shown below:

gcloud compute –service_version=”xyx1″ –project=”spark-project-1″ ssh –
-zone=”europe-west1-a” “spark-box-3g”

3. The next step should be the installation of Java. The following command can be used for this purpose:

sudo yum install java-1.7.0-openjdk-devel

To see the packages which are available for your download, just use the following command:

yum search java | grep ‘java-’

You should also ensure that Python, Java, and Scala have been installed.

                                       Checkout Apache Spark Interview Questions

4. The next step should involve the installation of Git. This can be done by executing the command given below:

yum install git

5. You can then use the “wget” command so as to obtain the necessary packages online.

6. You can then run the assembly sbt/sbt.

Once you have completed the above steps, you will be done and set to go.

About Author

NameRavindra Savaram
Author Bio


Ravindra Savaram is a Content Lead at Mindmajix.com. His passion lies in writing articles on the most popular IT platforms including Machine learning, DevOps, Data Science, Artificial Intelligence, RPA, Deep Learning, and so on. You can stay up to date on all these technologies by following him on LinkedIn and Twitter.