Home  >  Blog  >   Apche Spark

Installation of Spark on Google Compute Engine

Rating: 4

Spark On Google Compute Engine

GCE stands for Google Compute Engine. It is an infrastructure as a service which lets you execute your large scale computing workloads on the virtual machines which have been hosted at the infrastructure for Google. It is also possible for you to create your own and new machine and this will take you a very short time.

Are you intereted in taking up for Apache Spark Certification Training? Enroll for Free Demo on  Apache Spark Training!

The following are the installation steps:

1. Begin by creating a CentOS image on the GCE. The space used should be at least 3.8 GB, otherwise the system will not compile.

2. Once you are done, you can perform an ssh into the new machine. An example of how this can be done is shown below:

gcloud compute –service_version=”xyx1″ –project=”spark-project-1″ ssh –
-zone=”europe-west1-a” “spark-box-3g”

3. The next step should be the installation of Java. The following command can be used for this purpose:

sudo yum install java-1.7.0-openjdk-devel

To see the packages which are available for your download, just use the following command:

yum search java | grep ‘java-’

You should also ensure that Python, Java, and Scala have been installed.

MindMajix YouTube Channel

                                       Checkout Apache Spark Interview Questions

4. The next step should involve the installation of Git. This can be done by executing the command given below:

yum install git

5. You can then use the “wget” command so as to obtain the necessary packages online.

6. You can then run the assembly sbt/sbt.

Once you have completed the above steps, you will be done and set to go.

Join our newsletter

Stay updated with our newsletter, packed with Tutorials, Interview Questions, How-to's, Tips & Tricks, Latest Trends & Updates, and more ➤ Straight to your inbox!

Course Schedule
Apache Spark TrainingApr 27 to May 12View Details
Apache Spark TrainingApr 30 to May 15View Details
Apache Spark TrainingMay 04 to May 19View Details
Apache Spark TrainingMay 07 to May 22View Details
Last updated: 03 Apr 2023
About Author

Ravindra Savaram is a Technical Lead at Mindmajix.com. His passion lies in writing articles on the most popular IT platforms including Machine learning, DevOps, Data Science, Artificial Intelligence, RPA, Deep Learning, and so on. You can stay up to date on all these technologies by following him on LinkedIn and Twitter.

read more
Recommended Courses

1 / 15