Home  >  Blog  >   Apche Spark

Installation of Spark on Google Compute Engine

Rating: 4

Spark On Google Compute Engine

GCE stands for Google Compute Engine. It is an infrastructure as a service which lets you execute your large scale computing workloads on the virtual machines which have been hosted at the infrastructure for Google. It is also possible for you to create your own and new machine and this will take you a very short time.

Are you intereted in taking up for Apache Spark Certification Training? Enroll for Free Demo on  Apache Spark Training!

The following are the installation steps:

1. Begin by creating a CentOS image on the GCE. The space used should be at least 3.8 GB, otherwise the system will not compile.

2. Once you are done, you can perform an ssh into the new machine. An example of how this can be done is shown below:

gcloud compute –service_version=”xyx1″ –project=”spark-project-1″ ssh –
-zone=”europe-west1-a” “spark-box-3g”

3. The next step should be the installation of Java. The following command can be used for this purpose:

sudo yum install java-1.7.0-openjdk-devel

To see the packages which are available for your download, just use the following command:

yum search java | grep ‘java-’

You should also ensure that Python, Java, and Scala have been installed.

MindMajix YouTube Channel

                                       Checkout Apache Spark Interview Questions

4. The next step should involve the installation of Git. This can be done by executing the command given below:

yum install git

5. You can then use the “wget” command so as to obtain the necessary packages online.

6. You can then run the assembly sbt/sbt.

Once you have completed the above steps, you will be done and set to go.

Join our newsletter

Stay updated with our newsletter, packed with Tutorials, Interview Questions, How-to's, Tips & Tricks, Latest Trends & Updates, and more ➤ Straight to your inbox!

Course Schedule
Apache Spark TrainingOct 03 to Oct 18View Details
Apache Spark TrainingOct 07 to Oct 22View Details
Apache Spark TrainingOct 10 to Oct 25View Details
Apache Spark TrainingOct 14 to Oct 29View Details
Last updated: 30 September 2023
About Author

Ravindra Savaram is a Content Lead at Mindmajix.com. His passion lies in writing articles on the most popular IT platforms including Machine learning, DevOps, Data Science, Artificial Intelligence, RPA, Deep Learning, and so on. You can stay up to date on all these technologies by following him on LinkedIn and Twitter.

read more
Recommended Courses

1 / 15

Try NOW: One-Access by MindMajix