One of the features of Spark is that you can develop software on your local machine, and then later run it on a distributed system. This page describes how to set up Spark so that you can build and run on your laptop.
For compatibility, make sure you have the Java 8 JDK installed http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
You can also download a combined package with both NetBeans and the JDK http://www.oracle.com/technetwork/java/javase/downloads/jdk-netbeans-jsp-142931.html
Download Spark from http://spark.apache.org/downloads.html (Links to an external site.) Pick a directory where you want to keep your Spark installation, and unpack the downloaded .tgz file.
- Create your project with Category set to "Java", and "Project" set to "Java Application".
- After you finish creating your project, right click on the project name in the left-side Projects panel, and select "Properties" from the menu.
- In the Properties dialog:
- Select the Category "Libraries". Make sure that Java Platform is set to "JDK 1.8". Select "Compile" if not already selected.
Click on "Add JAR/Folder". In the dialog, navigate to the "jars" folder in your Spark installation directory. Select all the .jar files (click on the first one, shift-click on the last one to select all). Make sure that "Absolute Path" is selected.
Click "Choose" and then "OK".
Your project is now ready to compile and run Spark Java programs.
- To set the program arguments for running your Spark program, go back to the project Properties dialog, select "Run",
- Set up for Eclipse is very similar as NetBeans.
- Create a new project and give it a name and select "JAVA SE 1.8" or "JDK 1.8" for "Execution Environment JRE"
- Right click on the project name, select "Build Path" and then "Add External Archives"
- In the dialog, navigate to the "jars" folder in your Spark installation directory
- Select all the .jar files (click on the first one, shift-click on the last one to select all).
- Your project is now ready to run Spark Java programs
- To set the program arguments, right click on project's name and go to "Properties" -> "Run/Debug Settings"
- Select your main class and click "edit". In the dialog, click the "Arguments" tab and type your arguments separated by space in the "Arguments" box