Please follow following instructions on installation Apache Spark on Windows 10.
Please ensure that you have installed JDK 1.8 or above on your environment.
Installation of Scala 2.12.2
- Please Install Scala after downloading it.
- Scala can be downloaded from here.
- Download will give you a .msi file. Follow instructions and install Scala
Installation of Spark
- Spark Can be downloaded from here
- I am choosing version 2.1.1 prebuit for Hadoop. Please note, I shall be running this without Hadoop.
- Extract the tar file into a folder called c:\Spark
- The contents of the Extract will look like
- Download Winutils from these links : 64 bits
- Create a folder c:\Spark\Winutils\bin and copy this winutils.exe there
- The folder structure will look like
Setup Environment Variables
- Following environment variables will need to be setup:
- JAVA_HOME: C:\jdk1.8.0_91
- SCALA_HOME: C:\Program Files (x86)\scala\bin
- _JAVA_OPTION: -Xms128m -Xmx256m
- HADOOP_HOME: C:\Spark\WinUtils
- SPARK_HOME: C:\Program Files (x86)\scala\bin
- Create a folder c:\tmp\hive and give it read/write/execute privileges for all
- Navigate to SPARK_HOME/bin and execute command scala-shell