You have found the right place if you are looking for a step-by-step complete guide to setup Apache storm from scratch on your local machine and deploy a topology jar onto the same.
Step 1 : Setting up Installation Directory
Create a new directory ‘DevApps’ to install all required applications within the directory of your choice. This new directory would be referred as $INSTALL_DIR hereafter.
Tip for Windows UsersThere should not be any space in the full path of this directory.
If Windows is installed on C drive then this directory should look like – C:/DevApps.
Step 2 : Installing required applications
1. Java 8 SE Development Kit – 1.8.0_121
Download from here and install in $INSTALL_DIR
2. Python 2.7.13
Download from here and select $INSTALL_DIR for installation.
3. Apache Zookeeper – 3.4.9
Download from here and just extract to $INSTALL_DIR to install.
4. Apache Storm – 1.0.3
Download from here and extract to $INSTALL_DIR
Step 3 : Setting up Environment Variables
Add/Update following variables as below. Please note the version of Java SDK and Apache Storm that you have used for installation and make changes accordingly if needed.
JAVA_HOME = C:/DevApps/Java/jdk1.8.0_121 STORM_HOME = C:/DevApps/apache-storm-1.0.3
Add following to the existing PATH variable
PATH = %STORM_HOME%/bin;%JAVA_HOME/%bin;C:/DevApps/Python27;C:/DevApps/Python27/Lib/site-packages;C:/DevApps/Python27/Scripts;
[MacOS or Unix]
export STORM_HOME = ~/DevApps/apache-storm-1.0.3
Step 4 : Configuring local storm cluster
Create another folder storm-local in DevApps to be used as Storm local directory and use it in the next step.
Make following changes to storm.yaml file from %STORM_HOME%/conf folder
storm.zookeeper.servers: - "localhost" nimbus.host: "localhost" storm.local.dir: "C:/DevApps/storm-local"
storm.zookeeper.servers: - "localhost" nimbus.host: "localhost" storm.local.dir: "~/DevApps/storm-local"
Create a working config for ZooKeeper using the priovided sample.
and Copy/Rename zoo_sample.cfg as zoo.cfg
Step 5 : Starting local storm cluster
Let’s first start Zookeeper
[MacOS or Linux]
cd ~/DevApps/zookeeper-3.4.9/bin ./zkServer.sh start
Start Nimbus, Supervisor and Storm UI
storm nimbus storm supervisor storm ui
[MacOS or Linux]
cd ~/DevApps/apache-storm-1.0.3/bin ./storm nimbus & ./storm supervisor & ./storm ui &
Verify the setup by browsing to following link using any browser of your choice.
Step 6 : Deploying Topology
The syntax of the command deploys the topology in the storm cluster is as below.
storm jar <absolute_path_of_jar> <fully_qualified_name_of_main_class> <name_of_topology>
Let’s take an example to understand this –
absolute_path_of_jar = ~workspace/storm-trade-processing/target/storm-trade-processing-1.0-SNAPSHOT.jar
fully_qualified_class_name = com.techysoul.apache.storm.tradeprocessing.app.TradeProcessingTopology
name_of_topology = TradeProcessingTopology
Following command deploys the topology in the storm cluster.
cd ~/DevApps/apache-storm-1.0.3/bin ./storm jar ~/workspace/storm-trade-processing/target/storm-trade-processing-1.0-SNAPSHOT.jar com.techysoul.apache.storm.tradeprocessing.app.TradeProcessingTopology TradeProcessingTopology
Step 7 : Graceful Exit
cd ~/DevApps/zookeeper/bin ./zkServer.sh status ./zkServer.sh stop ./zkServer.sh status
Please let me know if you get stuck at any step while following this tutorial. Happy Storming!
Share this Post