# Cluster

Cluster is a set a nodes comprising of servers, brokers, controllers and minions.

![Pinot cluster components](https://2688850955-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LtH6nl58DdnZnelPdTc%2F-M1c68aqq6AOJfeKiQKl%2F-M1c6a3x65Y6tIAX2eNL%2Fcomponents.jpg?alt=media\&token=1173ac92-6966-40cf-a77b-f0e8c45e71b9)

Pinot leverages [Apache Helix](http://helix.apache.org/) for cluster management. Helix is a cluster management framework to manage replicated, partitioned resources in a distributed system. Helix uses Zookeeper to store cluster state and metadata.

### Cluster components

Briefly, Helix divides nodes into three logical components based on their responsibilities

#### Participant

The nodes that host distributed, partitioned resources

#### Spectator

The nodes that observe the current state of each Participant and use that information to access the resources. Spectators are notified of state changes in the cluster (state of a participant, or that of a partition in a participant).

#### Controller

The node that observes and controls the Participant nodes. It is responsible for coordinating all transitions in the cluster and ensuring that state constraints are satisfied while maintaining cluster stability*.*

**Pinot Servers** are modeled as Participants, more details about server nodes can be found in [Server](https://docs.pinot.apache.org/release-0.4.0/basics/components/server).\
**Pinot Brokers** are modeled as Spectators, more details about broker nodes can be found in [Broker](https://docs.pinot.apache.org/release-0.4.0/basics/components/broker).\
**Pinot Controllers** are modeled as Controllers, more details about controller nodes can be found in [Controller](https://docs.pinot.apache.org/release-0.4.0/basics/components/controller).

### Logical view

Another way to visualize the cluster is a logical view, wherein a cluster contains [tenants](https://docs.pinot.apache.org/release-0.4.0/basics/components/tenant), tenants contain [tables](https://docs.pinot.apache.org/release-0.4.0/basics/components/table), and tables contain [segments](https://docs.pinot.apache.org/release-0.4.0/basics/components/segment).

![](https://2688850955-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LtH6nl58DdnZnelPdTc%2F-M1cGq_K6f87emCw4pkc%2F-M1cHtSieWXlok7s8_HW%2FClusterLogical.jpg?alt=media\&token=5a8dc566-b8a3-4e2c-9f01-b32392ab3e69)

## Setup a Pinot Cluster

Typically, there is only cluster per environment/data center. There is no needed to create multiple Pinot clusters since Pinot supports the concept of [tenants](https://docs.pinot.apache.org/release-0.4.0/basics/components/tenant). At LinkedIn, the largest Pinot cluster consists of 1000+ nodes.

To setup a Pinot cluster, we need to first start Zookeeper.

{% tabs %}
{% tab title="Using docker images" %}

### 0. Create a Network

Create an isolated bridge network in docker

```
docker network create -d bridge pinot-demo
```

### 1. Start Zookeeper

Start Zookeeper in daemon.

```
docker run \
    --network=pinot-demo \
    --name pinot-zookeeper \
    --restart always \
    -p 2181:2181 \
    -d zookeeper:3.5.6
```

### 2. Start Zookeeper UI

Start [ZKUI](https://github.com/DeemOpen/zkui) to browse Zookeeper data at <http://localhost:9090>.

```
docker run \
    --network pinot-demo --name=zkui \
    -p 9090:9090 \
    -e ZK_SERVER=pinot-zookeeper:2181 \
    -d qnib/plain-zkui:latest
```

{% endtab %}

{% tab title="Using launcher scripts" %}
Download Pinot Distribution using instructions in [Download](https://apache-pinot.gitbook.io/apache-pinot-cookbook/getting-started/running-pinot-locally#download)

### 1. Start Zookeeper

```
bin/pinot-admin.sh StartZookeeper -zkPort 2181
```

### 2. Start Zooinspector

Install [zooinspector](https://github.com/jfim/zooinspector) to view the data in Zookeeper, and connect to localhost:2181
{% endtab %}
{% endtabs %}

Once we've started Zookeeper, we can start other components to join this cluster. If you're using docker, pull the latest `apachepinot/pinot` image.

{% tabs %}
{% tab title="Using docker images" %}

### Pull pinot docker image

You can try out pre-built Pinot all-in-one docker image.

```
export PINOT_VERSION=0.3.0-SNAPSHOT
export PINOT_IMAGE=apachepinot/pinot:${PINOT_VERSION}
docker pull ${PINOT_IMAGE}
```

(Optional) You can also follow the instructions [here](https://docs.pinot.apache.org/release-0.4.0/operators/tutorials/build-docker-images) to build your own images.
{% endtab %}
{% endtabs %}

To start other components to join the cluster

1. [Start Controller](https://docs.pinot.apache.org/release-0.4.0/basics/controller#starting-a-controller)
2. [Start Broker](https://docs.pinot.apache.org/release-0.4.0/basics/broker#starting-a-broker)
3. [Start Server](https://docs.pinot.apache.org/release-0.4.0/basics/server#starting-a-server)

Explore your cluster via [Pinot Data Explorer](https://docs.pinot.apache.org/release-0.4.0/basics/features/exploring-pinot)