Running on AWS
This guide provides a quick start for running Pinot on Amazon Web Services (AWS).
This document provides the basic instruction to set up a Kubernetes Cluster on Amazon Elastic Kubernetes Service (Amazon EKS)

1. Tooling Installation

1.1 Install Kubectl

Please follow this link (https://kubernetes.io/docs/tasks/tools/install-kubectl) to install kubectl.
For Mac User
1
brew install kubernetes-cli
Copied!
Please check kubectl version after installation.
1
kubectl version
Copied!
QuickStart scripts are tested under kubectl client version v1.16.3 and server version v1.13.12

1.2 Install Helm

Please follow this link (https://helm.sh/docs/using_helm/#installing-helm) to install helm.
For Mac User
1
brew install kubernetes-helm
Copied!
Please check helm version after installation.
1
helm version
Copied!
This QuickStart provides helm supports for helm v3.0.0 and v2.12.1. Please pick the script based on your helm version.

1.3 Install AWS CLI

For Mac User
1
curl "https://d1vvhvl2y92vvt.cloudfront.net/awscli-exe-macos.zip" -o "awscliv2.zip"
2
unzip awscliv2.zip
3
sudo ./aws/install
Copied!

1.4 Install Eksctl

For Mac User
1
brew tap weaveworks/tap
2
brew install weaveworks/tap/eksctl
Copied!

2. (Optional) Login to your AWS account.

For first time AWS user, please register your account at https://aws.amazon.com/.
Once created the account, you can go to AWS Identity and Access Management (IAM) to create a user and create access keys under Security Credential tab.
1
aws configure
Copied!
Environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY will override AWS configuration stored in file ~/.aws/credentials

3. (Optional) Create a Kubernetes cluster(EKS) in AWS

The script below will create a 1 node cluster named pinot-quickstart in us-west-2 with a t3.xlarge machine for demo purposes:
1
EKS_CLUSTER_NAME=pinot-quickstart
2
eksctl create cluster \
3
--name ${EKS_CLUSTER_NAME} \
4
--version 1.16 \
5
--region us-west-2 \
6
--nodegroup-name standard-workers \
7
--node-type t3.xlarge \
8
--nodes 1 \
9
--nodes-min 1 \
10
--nodes-max 1 \
11
--node-ami auto
Copied!
You can monitor the cluster status via this command:
1
EKS_CLUSTER_NAME=pinot-quickstart
2
aws eks describe-cluster --name ${EKS_CLUSTER_NAME}
Copied!
Once the cluster is in ACTIVE status, it's ready to be used.

4. Connect to an existing cluster

Simply run below command to get the credential for the cluster pinot-quickstart that you just created or your existing cluster.
1
EKS_CLUSTER_NAME=pinot-quickstart
2
aws eks update-kubeconfig --name ${EKS_CLUSTER_NAME}
Copied!
To verify the connection, you can run:
1
kubectl get nodes
Copied!

5. Pinot Quickstart

Please follow this Kubernetes QuickStart to deploy your Pinot Demo.

6. Delete a Kubernetes Cluster

1
EKS_CLUSTER_NAME=pinot-quickstart
2
aws eks delete-cluster --name ${EKS_CLUSTER_NAME}
Copied!
Last modified 10mo ago