arrow-left

All pages
gitbookPowered by GitBook
1 of 4

Loading...

Loading...

Loading...

Loading...

Running on Azure

This quickstart guide helps you get started running Pinot on Microsoft Azure.

In this quickstart guide, you will set up a Kubernetes Cluster on Azure Kubernetes Service (AKS)arrow-up-right

hashtag
1. Tooling Installation

hashtag
1.1 Install Kubectl

Follow this link () to install kubectl.

For Mac users

Check kubectl version after installation.

circle-info

Quickstart scripts are tested under kubectl client version v1.16.3 and server version v1.13.12

hashtag
1.2 Install Helm

To install Helm, see .

For Mac users

Check helm version after installation.

circle-info

This quickstart provides helm supports for helm v3.0.0 and v2.12.1. Pick the script based on your helm version.

hashtag
1.3 Install Azure CLI

Follow this link () to install Azure CLI.

For Mac users

hashtag
2. (Optional) Log in to your Azure account

This script will open your default browser to sign-in to your Azure Account.

hashtag
3. (Optional) Create a Resource Group

Use the following script create a resource group in location eastus.

hashtag
4. (Optional) Create a Kubernetes cluster(AKS) in Azure

This script will create a 3 node cluster named pinot-quickstart for demo purposes.

Modify the parameters in the following example command with your resource group and cluster details:

Once the command succeeds, the cluster is ready to be used.

hashtag
5. Connect to an existing cluster

Run the following command to get the credential for the cluster pinot-quickstart that you just created:

To verify the connection, run the following:

hashtag
6. Pinot quickstart

Follow this to deploy your Pinot demo.

hashtag
7. Delete a Kubernetes Cluster

https://kubernetes.io/docs/tasks/tools/install-kubectlarrow-up-right
Installing Helmarrow-up-right
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latestarrow-up-right
Kubernetes quickstart
brew install kubernetes-cli
kubectl version
brew install kubernetes-helm
helm version
brew update && brew install azure-cli
az login
AKS_RESOURCE_GROUP=pinot-demo
AKS_RESOURCE_GROUP_LOCATION=eastus
az group create --name ${AKS_RESOURCE_GROUP} \
                --location ${AKS_RESOURCE_GROUP_LOCATION}
AKS_RESOURCE_GROUP=pinot-demo
AKS_CLUSTER_NAME=pinot-quickstart
az aks create --resource-group ${AKS_RESOURCE_GROUP} \
              --name ${AKS_CLUSTER_NAME} \
              --node-count 3
AKS_RESOURCE_GROUP=pinot-demo
AKS_CLUSTER_NAME=pinot-quickstart
az aks get-credentials --resource-group ${AKS_RESOURCE_GROUP} \
                       --name ${AKS_CLUSTER_NAME}
kubectl get nodes
AKS_RESOURCE_GROUP=pinot-demo
AKS_CLUSTER_NAME=pinot-quickstart
az aks delete --resource-group ${AKS_RESOURCE_GROUP} \
              --name ${AKS_CLUSTER_NAME}

Running on public clouds

This page links to multiple quick start guides for deploying Pinot to different public cloud providers.

These quickstart guides show you how to run an Apache Pinot cluster using Kubernetes on different public cloud providers.

Running on Azurechevron-right
Running on GCPchevron-right
Running on AWSchevron-right

Running on GCP

This quickstart guide helps you get started running Pinot on Google Cloud Platform (GCP).

In this quickstart guide, you will set up a Kubernetes Cluster on Google Kubernetes Engine(GKE)arrow-up-right

hashtag
1. Tooling Installation

hashtag
1.1 Install Kubectl

Follow this link () to install kubectl.

For Mac users

Check kubectl version after installation.

circle-info

Quickstart scripts are tested under kubectl client version v1.16.3 and server version v1.13.12

hashtag
1.2 Install Helm

Follow this link () to install helm.

For Mac users

Check helm version after installation.

circle-info

This quickstart provides helm supports for helm v3.0.0 and v2.12.1. Choose the script based on your helm version.

hashtag
1.3 Install Google Cloud SDK

To install Google Cloud SDK, see

hashtag
1.3.1 For Mac users

  • Install Google Cloud SDK

Restart your shell

hashtag
2. (Optional) Initialize Google Cloud Environment

hashtag
3. (Optional) Create a Kubernetes cluster(GKE) in Google Cloud

This script will create a 3 node cluster named pinot-quickstart in us-west1-b with n1-standard-2 machines for demo purposes.

Modify the parameters in the following example command with your gcloud details:

Use the following command do monitor cluster status:

Once the cluster is in RUNNING status, it's ready to be used.

hashtag
4. Connect to an existing cluster

Run the following command to get the credential for the cluster pinot-quickstart that you just created:

To verify the connection, run the following:

hashtag
5. Pinot quickstart

Follow this to deploy your Pinot demo.

hashtag
6. Delete a Kubernetes Cluster

Running on AWS

This quickstart guide helps you get started running Pinot on Amazon Web Services (AWS).

In this quickstart guide, you will set up a Kubernetes Cluster on

hashtag
1. Tooling Installation

hashtag

https://kubernetes.io/docs/tasks/tools/install-kubectlarrow-up-right
https://helm.sh/docs/using_helm/#installing-helmarrow-up-right
Install the gcloud CLIarrow-up-right
Kubernetes quickstart
1.1 Install Kubectl

To install kubectl, see Install kubectlarrow-up-right.

For Mac users

Check kubectl version after installation.

circle-info

Quickstart scripts are tested under kubectl client version v1.16.3 and server version v1.13.12

hashtag
1.2 Install Helm

Follow this link (https://helm.sh/docs/using_helm/#installing-helmarrow-up-right) to install helm.

For Mac users

Check helm version after installation.

circle-info

This quickstart provides helm supports for helm v3.0.0 and v2.12.1. Pick the script based on your helm version.

hashtag
1.3 Install AWS CLI

Follow this link (https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html#install-tool-bundledarrow-up-right) to install AWS CLI.

For Mac users

hashtag
1.4 Install Eksctl

Follow this link (https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html#installing-eksctlarrow-up-right) to install AWS CLI.

For Mac users

hashtag
2. (Optional) Log in to your AWS account

For first-time AWS users, register your account at https://aws.amazon.com/arrow-up-right.

Once you have created the account, go to AWS Identity and Access Management (IAM)arrow-up-right to create a user and create access keys under Security Credential tab.

circle-info

Environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY will override the AWS configuration stored in file ~/.aws/credentials

hashtag
3. (Optional) Create a Kubernetes cluster(EKS) in AWS

The script below will create a 1 node cluster named pinot-quickstart in us-west-2 with a t3.xlarge machine for demo purposes:

For k8s 1.23+, run the following commands to allow the containers to provision their storage:

Use the following command to monitor the cluster status:

Once the cluster is in ACTIVE status, it's ready to be used.

hashtag
4. Connect to an existing cluster

Run the following command to get the credential for the cluster pinot-quickstart that you just created:

To verify the connection, run the following:

hashtag
5. Pinot quickstart

Follow this Kubernetes quickstart to deploy your Pinot demo.

hashtag
6. Delete a Kubernetes Cluster

Amazon Elastic Kubernetes Service (Amazon EKS)arrow-up-right
brew install kubernetes-cli
kubectl version
brew install kubernetes-helm
helm version
curl https://sdk.cloud.google.com | bash
exec -l $SHELL
gcloud init
GCLOUD_PROJECT=[your gcloud project name]
GCLOUD_ZONE=us-west1-b
GCLOUD_CLUSTER=pinot-quickstart
GCLOUD_MACHINE_TYPE=n1-standard-2
GCLOUD_NUM_NODES=3
gcloud container clusters create ${GCLOUD_CLUSTER} \
  --num-nodes=${GCLOUD_NUM_NODES} \
  --machine-type=${GCLOUD_MACHINE_TYPE} \
  --zone=${GCLOUD_ZONE} \
  --project=${GCLOUD_PROJECT}
gcloud compute instances list
GCLOUD_PROJECT=[your gcloud project name]
GCLOUD_ZONE=us-west1-b
GCLOUD_CLUSTER=pinot-quickstart
gcloud container clusters get-credentials ${GCLOUD_CLUSTER} --zone ${GCLOUD_ZONE} --project ${GCLOUD_PROJECT}
kubectl get nodes
GCLOUD_ZONE=us-west1-b
gcloud container clusters delete pinot-quickstart --zone=${GCLOUD_ZONE}
brew install kubernetes-cli
kubectl version
brew install kubernetes-helm
helm version
curl "https://d1vvhvl2y92vvt.cloudfront.net/awscli-exe-macos.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
brew tap weaveworks/tap
brew install weaveworks/tap/eksctl
aws configure
EKS_CLUSTER_NAME=pinot-quickstart
eksctl create cluster \
--name ${EKS_CLUSTER_NAME} \
--version 1.16 \
--region us-west-2 \
--nodegroup-name standard-workers \
--node-type t3.xlarge \
--nodes 1 \
--nodes-min 1 \
--nodes-max 1
eksctl utils associate-iam-oidc-provider --region=us-east-2 --cluster=pinot-quickstart --approve

eksctl create iamserviceaccount \
  --name ebs-csi-controller-sa \
  --namespace kube-system \
  --cluster pinot-quickstart \
  --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy \
  --approve \
  --role-only \
  --role-name AmazonEKS_EBS_CSI_DriverRole

eksctl create addon --name aws-ebs-csi-driver --cluster pinot-quickstart --service-account-role-arn arn:aws:iam::$(aws sts get-caller-identity --query Account --output text):role/AmazonEKS_EBS_CSI_DriverRole --force
EKS_CLUSTER_NAME=pinot-quickstart
aws eks describe-cluster --name ${EKS_CLUSTER_NAME} --region us-west-2
EKS_CLUSTER_NAME=pinot-quickstart
aws eks update-kubeconfig --name ${EKS_CLUSTER_NAME}
kubectl get nodes
EKS_CLUSTER_NAME=pinot-quickstart
aws eks delete-cluster --name ${EKS_CLUSTER_NAME}