Home  >  Blog  >   Apche Spark  > 

Installation of Spark on Google Compute Engine

Rating: 4

Spark On Google Compute Engine

GCE stands for Google Compute Engine. It is an infrastructure as a service which lets you execute your large scale computing workloads on the virtual machines which have been hosted at the infrastructure for Google. It is also possible for you to create your own and new machine and this will take you a very short time.

Are you intereted in taking up for Apache Spark Certification Training? Enroll for Free Demo on  Apache Spark Training!

The following are the installation steps:

1. Begin by creating a CentOS image on the GCE. The space used should be at least 3.8 GB, otherwise the system will not compile.

2. Once you are done, you can perform an ssh into the new machine. An example of how this can be done is shown below:

gcloud compute –service_version=”xyx1″ –project=”spark-project-1″ ssh –
-zone=”europe-west1-a” “spark-box-3g”

3. The next step should be the installation of Java. The following command can be used for this purpose:

sudo yum install java-1.7.0-openjdk-devel

To see the packages which are available for your download, just use the following command:

yum search java | grep ‘java-’

You should also ensure that Python, Java, and Scala have been installed.

MindMajix YouTube Channel

                                       Checkout Apache Spark Interview Questions

4. The next step should involve the installation of Git. This can be done by executing the command given below:

yum install git

5. You can then use the “wget” command so as to obtain the necessary packages online.

6. You can then run the assembly sbt/sbt.

Once you have completed the above steps, you will be done and set to go.

Join our newsletter

Stay updated with our newsletter, packed with Tutorials, Interview Questions, How-to's, Tips & Tricks, Latest Trends & Updates, and more ➤ Straight to your inbox!

Course Schedule
Apache Spark TrainingJun 03 to Jun 18
Apache Spark TrainingJun 06 to Jun 21
Apache Spark TrainingJun 10 to Jun 25
Apache Spark TrainingJun 13 to Jun 28
Last updated: 31 May 2023
About Author
Remy Sharp
Ravindra Savaram

Ravindra Savaram is a Content Lead at Mindmajix.com. His passion lies in writing articles on the most popular IT platforms including Machine learning, DevOps, Data Science, Artificial Intelligence, RPA, Deep Learning, and so on. You can stay up to date on all these technologies by following him on LinkedIn and Twitter.

Recommended Courses

1 /15