src/main/javafolder where the code resides and
src/test/javawhere the unit tests corresponding to the module’s code reside.
pinot-commonprovides classes common to Pinot components. Some key classes you will find here are:
config: Definitions for various elements of Pinot’s table config.
metrics: Definitions for base metrics provided by Controller, Broker and Server.
metadata: Definitions of metadata stored in Zookeeper.
pql.parsers: Code to compile PQL strings into corresponding AbstractSyntaxTrees (AST).
request: Autogenerated thrift classes representing various parts of PQL requests.
response: Definitions of response format returned by the Broker.
filesystem: provides abstractions for working with
segmentson local or remote filesystems. This module allows for users to plugin filesystems specific to their usecase. Extensions to the base
PinotFSshould ideally be housed in their specific modules so as not pull in unnecessary dependencies for all users.
pinot-transportmodule provides classes required to handle scatter-gather on Pinot Broker and netty wrapper classes used by Server to handle connections from Broker.
pinot-coremodules provides the core functionality of Pinot, specifically for handling segments, various index structures, query execution - filters, transformations, aggregations etc and support for realtime segments.
pinot-serverprovides server specific functionality including server startup and REST APIs exposed by the server.
pinot-controllerhouses all the controller specific functionality, including many cluster administration APIs, segment upload (for both offline and realtime), segment assignment, retention strategies etc.
pinot-brokerprovides broker functionality that includes wiring the broker startup sequence, building broker routing tables, PQL request handling.
pinot-minionprovides functionality for running auxiliary/periodic tasks on a Pinot Cluster such as purging records for compliance with regulations like GDPR.
pinot-hadoopprovides classes for segment generation jobs using Hadoop infrastructure.
pinot-tools: This module is a collection of many tools useful for setting up Pinot cluster, creating/updating segments.It also houses the Pinot quick start guide code.
pinot-perf: This module has a collection of benchmark test code used to evaluate design options.
pinot-integration-tests: This module holds integration tests that test functionality across multiple classes or components.
pinot-azure-filesystemare module added to support extensions to Pinot filesystem. The functionality is broken down into modules of their own to avoid polluting the common modules with additional large libraries. These libraries bring in transitive dependencies of their own that can cause classpath conflicts at runtime. We would like to avoid this for the common usage of Pinot as much as possible.