Manual cluster setup
This quick start guide will show you how to set up a Pinot cluster manually.
Start Pinot components (scripts or Docker images)
A manual cluster setup consists of the following components - 1. Zookeeper 2. Controller 3. Broker 4. Server 5. Kafka
We will run each of these components in separate containers
Start Pinot Components using docker
Prerequisites
If running locally, please ensure your docker cluster has enough resources, below is a sample config.
Pull docker image
You can try out the pre-built Pinot all-in-one docker image.
(Optional) You can also follow the instructions here to build your own images.
0. Create a Network
Create an isolated bridge network in docker
1. Start Zookeeper
Start Zookeeper in daemon mode. This is a single node zookeeper setup. Zookeeper is the central metadata store for Pinot and should be set up with replication for production use. For more information, see Running Replicated Zookeeper.
2. Start Pinot Controller
Start Pinot Controller in daemon and connect to Zookeeper.
The command below expects a 4GB memory container. Please tune-Xms
and-Xmx
if your machine doesn't have enough resources.
3. Start Pinot Broker
Start Pinot Broker in daemon and connect to Zookeeper.
The command below expects a 4GB memory container. Please tune-Xms
and-Xmx
if your machine doesn't have enough resources.
4. Start Pinot Server
Start Pinot Server in daemon and connect to Zookeeper.
The command below expects a 16GB memory container. Please tune-Xms
and-Xmx
if your machine doesn't have enough resources.
5. Start Kafka
Optionally, you can also start Kafka for setting up realtime streams. This brings up the Kafka broker on port 9092.
Now all Pinot related components are started as an empty cluster.
You can run the below command to check container status.
Sample Console Output
Run docker-compose up
to launch all the components.
You can run the below command to check container status.
Sample Console Output
Now it's time to start adding data to the cluster. Check out some of the Recipes or follow the Batch upload sample data and Stream sample data for instructions on loading your own data.
Last updated