Apache Pinot Docs
Search…
Running on AWS
This guide provides a quick start for running Pinot on Amazon Web Services (AWS).
This document provides the basic instruction to set up a Kubernetes Cluster on Amazon Elastic Kubernetes Service (Amazon EKS)​
1. Tooling Installation
1.1 Install Kubectl
Please follow this link (https://kubernetes.io/docs/tasks/tools/install-kubectl) to install kubectl.
For Mac User
1
brew install kubernetes-cli
Copied!
Please check kubectl version after installation.
1
kubectl version
Copied!
QuickStart scripts are tested under kubectl client version v1.16.3 and server version v1.13.12
1.2 Install Helm
For Mac User
1
brew install kubernetes-helm
Copied!
Please check helm version after installation.
1
helm version
Copied!
This QuickStart provides helm supports for helm v3.0.0 and v2.12.1. Please pick the script based on your helm version.
1.3 Install AWS CLI
​
Please follow this link (https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html#install-tool-bundled) to install AWS CLI.
For Mac User
1
curl "https://d1vvhvl2y92vvt.cloudfront.net/awscli-exe-macos.zip" -o "awscliv2.zip"
2
unzip awscliv2.zip
3
sudo ./aws/install
Copied!
1.4 Install Eksctl
Please follow this link (https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html#installing-eksctl) to install AWS CLI.
For Mac User
1
brew tap weaveworks/tap
2
brew install weaveworks/tap/eksctl
Copied!
2. (Optional) Login to your AWS account.
Once created the account, you can go to AWS Identity and Access Management (IAM) to create a user and create access keys under Security Credential tab.
1
aws configure
Copied!
Environment variables
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY will override AWS configuration stored in file ~/.aws/credentials3. (Optional) Create a Kubernetes cluster(EKS) in AWS
The script below will create a 1 node cluster named pinot-quickstart in us-west-2 with a t3.xlarge machine for demo purposes:
1
EKS_CLUSTER_NAME=pinot-quickstart
2
eksctl create cluster \
3
--name ${EKS_CLUSTER_NAME} \
4
--version 1.16 \
5
--region us-west-2 \
6
--nodegroup-name standard-workers \
7
--node-type t3.xlarge \
8
--nodes 1 \
9
--nodes-min 1 \
10
--nodes-max 1
Copied!
You can monitor the cluster status via this command:
1
EKS_CLUSTER_NAME=pinot-quickstart
2
aws eks describe-cluster --name ${EKS_CLUSTER_NAME} --region us-west-2
Copied!
Once the cluster is in ACTIVE status, it's ready to be used.
4. Connect to an existing cluster
Simply run below command to get the credential for the cluster pinot-quickstart that you just created or your existing cluster.
1
EKS_CLUSTER_NAME=pinot-quickstart
2
aws eks update-kubeconfig --name ${EKS_CLUSTER_NAME}
Copied!
To verify the connection, you can run:
1
kubectl get nodes
Copied!
5. Pinot Quickstart
6. Delete a Kubernetes Cluster
1
EKS_CLUSTER_NAME=pinot-quickstart
2
aws eks delete-cluster --name ${EKS_CLUSTER_NAME}
Copied!
Last modified 5mo ago