Hello Samza High Level API - Zookeeper Deployment
The hello-samza project is designed to get started with your first Samza job. In this tutorial, we will learn how to run a Samza application using ZooKeeper deployment model.
Get the Code
Let’s get started by cloning the hello-samza project
The project comes up with numerous examples and for this tutorial, we will pick the Wikipedia application.
Setting up the Deployment Environment
This command will download, install, and start ZooKeeper and Kafka. It will also check out the latest version of Samza and build it. All package files will be put in a sub-directory called “deploy” inside hello-samza’s root folder.
If you get a complaint that JAVA_HOME is not set, then you’ll need to set it to the path where Java is installed on your system.
Building the Hello Samza Project
NOTE: if you are building from the latest branch of hello-samza project and want to use your local copy of samza, make sure that you run the following step from your local Samza project first
With the environment setup complete, let us move on to building the hello-samza project. Execute the following commands:
We are now all set to deploy the application locally.
Running the Wikipedia application
In order to run the application, we will use the run-wikipedia-zk-application script.
The above command executes the helper script which invokes the WikipediaZkLocalApplication main class with the appropriate job configurations as command line arguments. The main class is an application wrapper that initializes the application and passes it to the local runner for execution. It is blocking and waits for the LocalApplicationRunner to finish.
To run your own application using ZooKeeper deployment model, you would need something similar to WikipediaZkLocalApplication class that initializes your application and uses the LocalApplicationRunner to run it. To learn more about the internals checkout deployment-models documentation and the configurations table.
Getting back to our example, the application consumes a feed of real-time edits from Wikipedia, and produces them to a Kafka topic called “wikipedia-stats”. Give the job a minute to startup, and then tail the Kafka topic. To do so, run the following command:
The messages in the stats topic should look like the sample below:
Excellent! Now that the job is running, open the plan.html file under deploy/samza/bin directory to take a look at the execution plan for the Wikipedia application. The execution plan is a colorful graphic representing various stages of your application and how they are connected. Here is a sample plan visualization: