Running in Kubernetes

Pinot quickstart in Kubernetes

Get started running Pinot in Kubernetes.

Note: The examples in this guide are sample configurations to be used as reference. For production setup, you may want to customize it to your needs, especially enable those security features like TLS/Auth etc.

Prerequisites

Kubernetes

This guide assumes that you already have a running Kubernetes cluster.

If you haven't yet set up a Kubernetes cluster, see the links below for instructions:

• Enable Kubernetes on Docker-Desktop

• Install Minikube for local setup

  • Make sure to run with enough resources: minikube start --vm=true --cpus=4 --memory=8g --disk-size=50g

• Set up a Kubernetes Cluster using Amazon Elastic Kubernetes Service (Amazon EKS)

• Set up a Kubernetes Cluster using Google Kubernetes Engine (GKE)

• Set up a Kubernetes Cluster using Azure Kubernetes Service (AKS)

Pinot

Make sure that you've downloaded Apache Pinot. The scripts for the setup in this guide can be found in our open source project on GitHub.

Git clone project source

# checkout pinot
git clone https://github.com/apache/pinot.git
cd pinot/helm/pinot

Set up a Pinot cluster in Kubernetes

Start Pinot with Helm

Run Helm with pre-installed package

The Pinot repository has pre-packaged Helm charts for Pinot and Presto. The Helm repository index file is here.

helm repo add pinot https://raw.githubusercontent.com/apache/pinot/master/helm
kubectl create ns pinot-quickstart
helm install pinot pinot/pinot \
    -n pinot-quickstart \
    --set cluster.name=pinot \
    --set server.replicaCount=2

Note: Specify StorageClass based on your cloud vendor. Don't mount a blob store (such as AzureFile, GoogleCloudStorage, or S3) as the data serving file system. Use only Amazon EBS/GCP Persistent Disk/Azure Disk-style disks.

• For AWS: "gp2"

• For GCP: "pd-ssd" or "standard"

• For Azure: "AzureDisk"

• For Docker-Desktop: "hostpath"

Run Helm script within Git repo

Check Pinot deployment status

kubectl get all -n pinot-quickstart

Next steps

Once your cluster is running, load data into it:

Stream Ingestion (Kubernetes)

Connect query engines to your Pinot cluster:

Query Engines (Kubernetes)

Delete a Pinot cluster in Kubernetes

To delete your Pinot cluster in Kubernetes, run the following command:

kubectl delete ns pinot-quickstart