This page links to multiple quick start guides for deploying Pinot to different public cloud providers.
These quickstart guides show you how to run an Apache Pinot cluster using Kubernetes on different public cloud providers.
Running on AzureRunning on GCPRunning on AWSThis quickstart guide helps you get started running Pinot on Microsoft Azure.
In this quickstart guide, you will set up a Kubernetes Cluster on Azure Kubernetes Service (AKS)
Follow this link (https://kubernetes.io/docs/tasks/tools/install-kubectl) to install kubectl.
For Mac users
brew install kubernetes-cli
Check kubectl version after installation.
kubectl version
To install Helm, see Installing Helm.
For Mac users
brew install kubernetes-helm
Check helm version after installation.
helm version
Follow this link (https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest) to install Azure CLI.
For Mac users
brew update && brew install azure-cli
This script will open your default browser to sign-in to your Azure Account.
az login
Use the following script create a resource group in location eastus.
AKS_RESOURCE_GROUP=pinot-demo
AKS_RESOURCE_GROUP_LOCATION=eastus
az group create --name ${AKS_RESOURCE_GROUP} \
--location ${AKS_RESOURCE_GROUP_LOCATION}
This script will create a 3 node cluster named pinot-quickstart for demo purposes.
Modify the parameters in the following example command with your resource group and cluster details:
AKS_RESOURCE_GROUP=pinot-demo
AKS_CLUSTER_NAME=pinot-quickstart
az aks create --resource-group ${AKS_RESOURCE_GROUP} \
--name ${AKS_CLUSTER_NAME} \
--node-count 3
Once the command succeeds, the cluster is ready to be used.
Run the following command to get the credential for the cluster pinot-quickstart that you just created:
AKS_RESOURCE_GROUP=pinot-demo
AKS_CLUSTER_NAME=pinot-quickstart
az aks get-credentials --resource-group ${AKS_RESOURCE_GROUP} \
--name ${AKS_CLUSTER_NAME}
To verify the connection, run the following:
kubectl get nodes
Follow this Kubernetes quickstart to deploy your Pinot demo.
AKS_RESOURCE_GROUP=pinot-demo
AKS_CLUSTER_NAME=pinot-quickstart
az aks delete --resource-group ${AKS_RESOURCE_GROUP} \
--name ${AKS_CLUSTER_NAME}
This quickstart guide helps you get started running Pinot on Google Cloud Platform (GCP).
In this quickstart guide, you will set up a Kubernetes Cluster on Google Kubernetes Engine(GKE)
Follow this link (https://kubernetes.io/docs/tasks/tools/install-kubectl) to install kubectl.
For Mac users
brew install kubernetes-cli
Check kubectl version after installation.
kubectl version
Follow this link (https://helm.sh/docs/using_helm/#installing-helm) to install helm.
For Mac users
brew install kubernetes-helm
Check helm version after installation.
helm version
To install Google Cloud SDK, see Install the gcloud CLI
Install Google Cloud SDK
curl https://sdk.cloud.google.com | bash
Restart your shell
exec -l $SHELL
gcloud init
This script will create a 3 node cluster named pinot-quickstart in us-west1-b with n1-standard-2 machines for demo purposes.
Modify the parameters in the following example command with your gcloud details:
GCLOUD_PROJECT=[your gcloud project name]
GCLOUD_ZONE=us-west1-b
GCLOUD_CLUSTER=pinot-quickstart
GCLOUD_MACHINE_TYPE=n1-standard-2
GCLOUD_NUM_NODES=3
gcloud container clusters create ${GCLOUD_CLUSTER} \
--num-nodes=${GCLOUD_NUM_NODES} \
--machine-type=${GCLOUD_MACHINE_TYPE} \
--zone=${GCLOUD_ZONE} \
--project=${GCLOUD_PROJECT}
Use the following command do monitor cluster status:
gcloud compute instances list
Once the cluster is in RUNNING status, it's ready to be used.
Run the following command to get the credential for the cluster pinot-quickstart that you just created:
GCLOUD_PROJECT=[your gcloud project name]
GCLOUD_ZONE=us-west1-b
GCLOUD_CLUSTER=pinot-quickstart
gcloud container clusters get-credentials ${GCLOUD_CLUSTER} --zone ${GCLOUD_ZONE} --project ${GCLOUD_PROJECT}
To verify the connection, run the following:
kubectl get nodes
Follow this Kubernetes quickstart to deploy your Pinot demo.
GCLOUD_ZONE=us-west1-b
gcloud container clusters delete pinot-quickstart --zone=${GCLOUD_ZONE}
This quickstart guide helps you get started running Pinot on Amazon Web Services (AWS).
In this quickstart guide, you will set up a Kubernetes Cluster on Amazon Elastic Kubernetes Service (Amazon EKS)
To install kubectl, see Install kubectl.
For Mac users
brew install kubernetes-cli
Check kubectl version after installation.
kubectl version
Follow this link (https://helm.sh/docs/using_helm/#installing-helm) to install helm.
For Mac users
brew install kubernetes-helm
Check helm version after installation.
helm version
Follow this link (https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html#install-tool-bundled) to install AWS CLI.
For Mac users
curl "https://d1vvhvl2y92vvt.cloudfront.net/awscli-exe-macos.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
Follow this link (https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html#installing-eksctl) to install AWS CLI.
For Mac users
brew tap weaveworks/tap
brew install weaveworks/tap/eksctl
For first-time AWS users, register your account at https://aws.amazon.com/.
Once you have created the account, go to AWS Identity and Access Management (IAM) to create a user and create access keys under Security Credential tab.
aws configure
The script below will create a 1 node cluster named pinot-quickstart in us-west-2 with a t3.xlarge machine for demo purposes:
EKS_CLUSTER_NAME=pinot-quickstart
eksctl create cluster \
--name ${EKS_CLUSTER_NAME} \
--version 1.16 \
--region us-west-2 \
--nodegroup-name standard-workers \
--node-type t3.xlarge \
--nodes 1 \
--nodes-min 1 \
--nodes-max 1
For k8s 1.23+, run the following commands to allow the containers to provision their storage:
eksctl utils associate-iam-oidc-provider --region=us-east-2 --cluster=pinot-quickstart --approve
eksctl create iamserviceaccount \
--name ebs-csi-controller-sa \
--namespace kube-system \
--cluster pinot-quickstart \
--attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
--approve \
--role-only \
--role-name AmazonEKS_EBS_CSI_DriverRole
eksctl create addon --name aws-ebs-csi-driver --cluster pinot-quickstart --service-account-role-arn arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):role/AmazonEKS_EBS_CSI_DriverRole --force
Use the following command to monitor the cluster status:
EKS_CLUSTER_NAME=pinot-quickstart
aws eks describe-cluster --name ${EKS_CLUSTER_NAME} --region us-west-2
Once the cluster is in ACTIVE status, it's ready to be used.
Run the following command to get the credential for the cluster pinot-quickstart that you just created:
EKS_CLUSTER_NAME=pinot-quickstart
aws eks update-kubeconfig --name ${EKS_CLUSTER_NAME}
To verify the connection, run the following:
kubectl get nodes
Follow this Kubernetes quickstart to deploy your Pinot demo.
EKS_CLUSTER_NAME=pinot-quickstart
aws eks delete-cluster --name ${EKS_CLUSTER_NAME}