arrow-left

All pages
gitbookPowered by GitBook
1 of 1

Loading...

Running Pinot locally

This quick start guide will help you bootstrap a Pinot standalone instance on your local machine.

In this guide, you'll learn how to download and install Apache Pinot as a standalone instance.

  • Download Apache Pinot

  • Set up a cluster

hashtag
Download Apache Pinot

First, download the Pinot distribution for this tutorial. You can either download a packaged release or build a distribution from the source code.

hashtag
Prerequisites

  • Install with JDK 11 or 17. JDK 21 is still ongoing.

  • For JDK 8 support, Pinot 0.12.1 is the last version compilable from the source code.

  • Pinot 1.0+ doesn't support JDK 8 anymore, build with JDK 11+

Note that some installations of the JDK do not contain the JNI bindings necessary to run all tests. If you see an error like java.lang.UnsatisfiedLinkError while running tests, you might need to change your JDK.

If using Homebrew, install AdoptOpenJDK 11 using brew install --cask adoptopenjdk11.

circle-info

Support for M1 and M2 Mac systems

Currently, Apache Pinot doesn't provide official binaries for M1 or M2 Macs. For instructions, see .

Download the distribution or build from source by selecting one of the following tabs:

Download the latest binary release from , or use this command:

Extract the TAR file:

Navigate to the directory containing the launcher scripts:

You can also find older versions of Apache Pinot at . For example, to download Pinot 0.10.0, run the following command:

Follow these steps to checkout code from and build Pinot locally

hashtag
M1 and M2 Mac Support

Currently, Apache Pinot doesn't provide official binaries for M1 or M2 Mac systems. Follow the instructions below to run on an M1 or M2 Mac:

  1. Add the following to your ~/.m2/settings.xml:

  1. Install Rosetta:

hashtag
Set up a cluster

Now that we've downloaded Pinot, it's time to set up a cluster. There are two ways to do this: through quick start or through setting up a cluster manually.

hashtag
Quick start

Pinot comes with quick start commands that launch instances of Pinot components in the same process and import pre-built datasets.

For example, the following quick start command launches Pinot with a baseball dataset pre-loaded:

For a list of all the available quick start commands, see the .

hashtag
Manual cluster

If you want to play with bigger datasets (more than a few megabytes), you can launch each component individually.

The video below is a step-by-step walk through for launching the individual components of Pinot and scaling them to multiple instances.

You can find the commands that are shown in this video in the .

circle-info

The examples below assume that you are using Java 8.

If you are using Java 11+ users, remove the GC settings insideJAVA_OPTS. So, for example, instead of this:

Use the following:

hashtag
Start Zookeeper

You can use to browse the Zookeeper instance.

hashtag
Start Pinot Controller

hashtag
Start Pinot Broker

hashtag
Start Pinot Server

hashtag
Start Kafka

Once your cluster is up and running, you can head over to to learn how to run queries against the data.

hashtag
Start a Pinot component in debug mode with IntelliJ

Set break points and inspect variables by starting a Pinot component with debug mode in IntelliJ.

The following example demonstrates server debugging:

  1. First, startzookeeper , controller, and broker using the .

  2. Then, use the following configuration under $PROJECT_DIR$\.run ) to start the server, replacing the metrics-core version and cluster name as needed. This is an example of how to use it.

circle-info

Prerequisites

Install Apache Mavenarrow-up-right 3.6 or higher

circle-info

For M1 and M2 Macs, first follow the steps below first.

Check out Pinot:

Build Pinot:

circle-info

If you're building with JDK 8, add Maven option -Djdk.version=8.

Navigate to the directory containing the setup scripts. Note that Pinot scripts are located under pinot-distribution/target, not the target directory under root.

Pinot can also be installed on Mac OS using the Brew package manager. For instructions on installing Brew, see the Brew documentationarrow-up-right.

tar -zxvf apache-pinot-$PINOT_VERSION-bin.tar.gz
cd apache-pinot-$PINOT_VERSION-bin
OLDER_VERSION="0.10.0"
wget https://archive.apache.org/dist/pinot/apache-pinot-$OLDER_VERSION/apache-pinot-$OLDER_VERSION-bin.tar.gz
Start a Pinot component in debug mode with IntelliJ
M1 and M2 Mac Support
Apache Pinotarrow-up-right
https://archive.apache.org/dist/pinot/arrow-up-right
Githubarrow-up-right
Quick Start Examples
this Github repositoryarrow-up-right
Zooinspectorarrow-up-right
Exploring Pinot
steps described above
commitarrow-up-right
PINOT_VERSION=0.12.0 #set to the Pinot version you decide to use

wget https://downloads.apache.org/pinot/apache-pinot-$PINOT_VERSION/apache-pinot-$PINOT_VERSION-bin.tar.gz
git clone https://github.com/apache/pinot.git
cd pinot
mvn install package -DskipTests -Pbin-dist
cd build
brew install pinot
<settings>
  <activeProfiles>
    <activeProfile>
      apple-silicon
    </activeProfile>
  </activeProfiles>
  <profiles>
    <profile>
      <id>apple-silicon</id>
      <properties>
        <os.detected.classifier>osx-x86_64</os.detected.classifier>
      </properties>
    </profile>
  </profiles>
</settings>  
softwareupdate --install-rosetta
./bin/pinot-admin.sh QuickStart -type batch
export JAVA_OPTS="-Xms4G -Xmx8G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-controller.log"
export JAVA_OPTS="-Xms4G -Xmx8G"
./bin/pinot-admin.sh StartZookeeper \
  -zkPort 2191
export JAVA_OPTS="-Xms4G -Xmx8G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-controller.log"
./bin/pinot-admin.sh StartController \
    -zkAddress localhost:2191 \
    -controllerPort 9000
export JAVA_OPTS="-Xms4G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-broker.log"
./bin/pinot-admin.sh StartBroker \
    -zkAddress localhost:2191
export JAVA_OPTS="-Xms4G -Xmx16G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-server.log"
./bin/pinot-admin.sh StartServer \
    -zkAddress localhost:2191
./bin/pinot-admin.sh  StartKafka \ 
  -zkAddress=localhost:2191/kafka \
  -port 19092
<component name="ProjectRunConfigurationManager">
  <configuration default="false" name="HelixServerStarter" type="Application" factoryName="Application" nameIsGenerated="true">
    <classpathModifications>
      <entry path="$PROJECT_DIR$/pinot-plugins/pinot-metrics/pinot-yammer/target/classes" />
      <entry path="$MAVEN_REPOSITORY$/com/yammer/metrics/metrics-core/2.2.0/metrics-core-2.2.0.jar" />
    </classpathModifications>
    <option name="MAIN_CLASS_NAME" value="org.apache.pinot.server.starter.helix.HelixServerStarter" />
    <module name="pinot-server" />
    <extension name="coverage">
      <pattern>
        <option name="PATTERN" value="org.apache.pinot.server.starter.helix.*" />
        <option name="ENABLED" value="true" />
      </pattern>
    </extension>
    <method v="2">
      <option name="Make" enabled="true" />
    </method>
  </configuration>
</component>
Neha Pawar from the Apache Pinot team shows you how to set up a Pinot cluster