Running Pinot locally
This quick start guide will help you bootstrap a Pinot standalone instance on your local machine.
In this guide, you'll learn how to download and install Apache Pinot as a standalone instance.
First, download the Pinot distribution for this tutorial. You can either download a packaged release or build a distribution from the source code.
- Install JDK11 or higher (JDK16 is not yet supported).
- For JDK 8 support, use Pinot 0.7.1 or compile from the source code.
Note that some installations of the JDK do not contain the JNI bindings necessary to run all tests. If you see an error like
java.lang.UnsatisfiedLinkError while running tests, you might need to change your JDK. If using Homebrew, install AdoptOpenJDK 11 using
brew install --cask adoptopenjdk11.Support for M1 and M2 Mac systems
Currently, Apache Pinot doesn't provide official binaries for M1 or M2 Macs. For instructions, see M1 and M2 Mac Support.
Download the distribution or build from source by selecting one of the following tabs:
Download the release
Build from source
PINOT_VERSION=0.12.0 #set to the Pinot version you decide to use
wget https://downloads.apache.org/pinot/apache-pinot-$PINOT_VERSION/apache-pinot-$PINOT_VERSION-bin.tar.gz
Extract the TAR file:
tar -zxvf apache-pinot-$PINOT_VERSION-bin.tar.gz
Navigate to the directory containing the launcher scripts:
cd apache-pinot-$PINOT_VERSION-bin
You can also find older versions of Apache Pinot at https://archive.apache.org/dist/pinot/. For example, to download Pinot 0.10.0, run the following command:
OLDER_VERSION="0.10.0"
wget https://archive.apache.org/dist/pinot/apache-pinot-$OLDER_VERSION/apache-pinot-$OLDER_VERSION-bin.tar.gz
Prerequisites
Check out Pinot:
git clone https://github.com/apache/pinot.git
cd pinot
Build Pinot:
If you're building with JDK 8, add Maven option
-Djdk.version=8.mvn install package -DskipTests -Pbin-dist
Navigate to the directory containing the setup scripts. Note that Pinot scripts are located under
pinot-distribution/target, not the target directory under root.cd build
Currently, Apache Pinot doesn't provide official binaries for M1 or M2 Mac systems. Follow the instructions below to run on an M1 or M2 Mac:
- 1.Add the following to your
~/.m2/settings.xml:
<settings>
<activeProfiles>
<activeProfile>
apple-silicon
</activeProfile>
</activeProfiles>
<profiles>
<profile>
<id>apple-silicon</id>
<properties>
<os.detected.classifier>osx-x86_64</os.detected.classifier>
</properties>
</profile>
</profiles>
</settings>
- 2.Install Rosetta:
softwareupdate --install-rosetta
Now that we've downloaded Pinot, it's time to set up a cluster. There are two ways to do this: through quick start or through setting up a cluster manually.
Pinot comes with quick start commands that launch instances of Pinot components in the same process and import pre-built datasets.
For example, the following quick start command launches Pinot with a baseball dataset pre-loaded:
./bin/pinot-admin.sh QuickStart -type batch
If you want to play with bigger datasets (more than a few megabytes), you can launch each component individually.
The video below is a step-by-step walk through for launching the individual components of Pinot and scaling them to multiple instances.
Neha Pawar from the Apache Pinot team shows you how to set up a Pinot cluster
The examples below assume that you are using Java 8.
If you are using Java 11+ users, remove the GC settings inside
JAVA_OPTS. So, for example, instead of this:export JAVA_OPTS="-Xms4G -Xmx8G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-controller.log"
Use the following:
export JAVA_OPTS="-Xms4G -Xmx8G"
./bin/pinot-admin.sh StartZookeeper \
-zkPort 2191
export JAVA_OPTS="-Xms4G -Xmx8G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-controller.log"
./bin/pinot-admin.sh StartController \
-zkAddress localhost:2191 \
-controllerPort 9000
export JAVA_OPTS="-Xms4G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-broker.log"
./bin/pinot-admin.sh StartBroker \
-zkAddress localhost:2191
export JAVA_OPTS="-Xms4G -Xmx16G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-server.log"
./bin/pinot-admin.sh StartServer \
-zkAddress localhost:2191
./bin/pinot-admin.sh StartKafka \
-zkAddress=localhost:2191/kafka \
-port 19092
Once your cluster is up and running, you can head over to Exploring Pinot to learn how to run queries against the data.
Set break points and inspect variables by starting a Pinot component with debug mode in IntelliJ.
The following example demonstrates server debugging:
- 1.
- 2.Then, use the following configuration under
$PROJECT_DIR$\.run) to start the server, replacing themetrics-coreversion and cluster name as needed. This commit is an example of how to use it.
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="HelixServerStarter" type="Application" factoryName="Application" nameIsGenerated="true">
<classpathModifications>
<entry path="$PROJECT_DIR$/pinot-plugins/pinot-metrics/pinot-yammer/target/classes" />
<entry path="$MAVEN_REPOSITORY$/com/yammer/metrics/metrics-core/2.2.0/metrics-core-2.2.0.jar" />
</classpathModifications>
<option name="MAIN_CLASS_NAME" value="org.apache.pinot.server.starter.helix.HelixServerStarter" />
<module name="pinot-server" />
<extension name="coverage">
<pattern>
<option name="PATTERN" value="org.apache.pinot.server.starter.helix.*" />
<option name="ENABLED" value="true" />
</pattern>
</extension>
<method v="2">
<option name="Make" enabled="true" />
</method>
</configuration>
</component>
Last modified 7mo ago