arrow-left

All pages
gitbookPowered by GitBook
1 of 1

Loading...

Manual cluster setup

This quick start guide will show you how to set up a Pinot cluster manually.

hashtag
Start Pinot components (scripts or docker images)

hashtag
Start Pinot Components using docker

hashtag
Pull docker image

You can try out pre-built Pinot all-in-one docker image.

(Optional) You can also follow the instructions to build your own images.

hashtag
0. Create a Network

Create an isolated bridge network in docker

hashtag
1. Start Zookeeper

Start Zookeeper in daemon mode. This is a single node zookeeper setup. Zookeeper is the central metadata store for Pinot and should be set up with replication for production use. See for more information.

Start to browse Zookeeper data at .

Alternately, you can use .

hashtag
2. Start Pinot Controller

Start Pinot Controller in daemon and connect to Zookeeper.

hashtag
3. Start Pinot Broker

Start Pinot Broker in daemon and connect to Zookeeper.

hashtag
4. Start Pinot Server

Start Pinot Server in daemon and connect to Zookeeper.

hashtag
5. Start Kafka

Optionally, you can also start Kafka for setting up realtime streams. This brings up the Kafka broker on port 9092.

Now all Pinot related components are started as an empty cluster.

You can run below command to check container status.

Sample Console Output

circle-info

Prerequisites

Follow instruction in to get Pinot

hashtag

Now it's time to start adding data to the cluster. Check out some of the or follow the and for instructions on loading your own data.

Start Pinot components via launcher scripts

hashtag
1. Start Zookeeper

You can use Zooinspectorarrow-up-right to browse the Zookeeper instance.

hashtag
2. Start Pinot Controller

hashtag
3. Start Pinot Broker

hashtag
4. Start Pinot Server

hashtag
5. Start Kafka

Now all Pinot related components are started as an empty cluster.

export PINOT_VERSION=0.3.0-SNAPSHOT
export PINOT_IMAGE=apachepinot/pinot:${PINOT_VERSION}
docker pull ${PINOT_IMAGE}
docker network create -d bridge pinot-demo
docker run \
    --network=pinot-demo \
    --name  pinot-zookeeper \
    --restart always \
    -p 2181:2181 \
    -d zookeeper:3.5.6
here
https://zookeeper.apache.org/doc/r3.6.0/zookeeperStarted.html#sc_RunningReplicatedZooKeeperarrow-up-right
ZKUIarrow-up-right
http://localhost:9090arrow-up-right
Zooinspectorarrow-up-right
Getting Pinot
Recipes
Batch upload sample data
Stream sample data
docker run --rm -ti \
    --network pinot-demo --name=zkui \
    -p 9090:9090 \
    -e ZK_SERVER=pinot-zookeeper:2181 \
    -d qnib/plain-zkui:latest
docker run --rm -ti \
    --network=pinot-demo \
    --name pinot-controller \
    -p 9000:9000 \
    -d ${PINOT_IMAGE} StartController \
    -zkAddress pinot-zookeeper:2181
docker run --rm -ti \
    --network=pinot-demo \
    --name pinot-broker \
    -d ${PINOT_IMAGE} StartBroker \
    -zkAddress pinot-zookeeper:2181
docker run --rm -ti \
    --network=pinot-demo \
    --name pinot-server \
    -d ${PINOT_IMAGE} StartServer \
    -zkAddress pinot-zookeeper:2181
docker run --rm -ti \
    --network pinot-demo --name=kafka \
    -e KAFKA_ZOOKEEPER_CONNECT=pinot-zookeeper:2181/kafka \
    -e KAFKA_BROKER_ID=0 \
    -e KAFKA_ADVERTISED_HOST_NAME=kafka \
    -d wurstmeister/kafka:latest
docker container ls -a
CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS                                                  NAMES
9ec20e4463fa        wurstmeister/kafka:latest   "start-kafka.sh"         43 minutes ago      Up 43 minutes                                                              kafka
0775f5d8d6bf        apachepinot/pinot:latest    "./bin/pinot-admin.s…"   44 minutes ago      Up 44 minutes       8096-8099/tcp, 9000/tcp                                pinot-server
64c6392b2e04        apachepinot/pinot:latest    "./bin/pinot-admin.s…"   44 minutes ago      Up 44 minutes       8096-8099/tcp, 9000/tcp                                pinot-broker
b6d0f2bd26a3        apachepinot/pinot:latest    "./bin/pinot-admin.s…"   45 minutes ago      Up 45 minutes       8096-8099/tcp, 0.0.0.0:9000->9000/tcp                  pinot-controller
570416fc530e        zookeeper:3.5.6             "/docker-entrypoint.…"   45 minutes ago      Up 45 minutes       2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, 8080/tcp   pinot-zookeeper
cd apache-pinot-incubating-${PINOT_VERSION}-bin
bin/pinot-admin.sh StartZookeeper \
  -zkPort 2191
bin/pinot-admin.sh StartController \
    -zkAddress localhost:2191 \
    -controllerPort 9000
bin/pinot-admin.sh StartBroker \
    -zkAddress localhost:2191
bin/pinot-admin.sh StartServer \
    -zkAddress localhost:2191
bin/pinot-admin.sh  StartKafka \ 
  -zkAddress=localhost:2191/kafka \
  -port 19092