Monday, December 24, 2018

Install Apache Spark 2.0 on Amazon EC2 - Quick Setup


1.) Install java
     sudo yum install java-1.8.0-openjdk

2.) Download the apache spark
     wget http://mirror.rise.ph/apache/spark/spark-2.4.0/spark-2.4.0-bin-hadoop2.7.tgz

3.) Unzip it
    tar -xvf spark-2.4.0-bin-hadoop2.7.tgz

4.) Create a symbolic Link
     ln -s spark-2.4.0-bin-hadoop2.7 spark

5.) Edit the bashprofile
     sudo nano .bashrc
    
    Add the following codes:
    export SPARK_HOME=/home/ec2-user/spark 
    export PATH=$PATH:$SPARK_HOME

6.) Save changes with .bashrc
      . .bashrc

7.) Test spark
      spark-shell

0 comments:

Post a Comment