Table
The tables below shows the properties available to set at the table level.
Top-level fields
Second-level fields
The following properties can be nested inside the top-level configurations.
Quota
Routing
Find details on configuring routing here.
Query
Segments config
Table index config
This section is used to specify some general index configuration and multi-column indexes like Star-tree.
Before Pinot version 0.13, the configuration described above was also used to configure certain single-column indexes. While this approach is still supported, it is highly recommended to specify these indexes in the [Field Config List](#Field Config List) section instead. The documentation page for each index type provides guidance on how to utilize this section to create that specific index type. This updated method offers more flexibility and aligns with best practices for configuring single-column indexes in Pinot.
Field Config List
Specify the columns and the type of indices to be created on those columns.
Deprecated configuration options
There are several deprecated configuration options in Pinot that are still supported but recommended for migration to newer ways of configuration. Here's a summary of these options:
indexTypes
Description: An older way to define indexes enabled for a column.
Supported Index Types: Text, FST, Timestamp, H3 (also known as geospatial).
Note: Some index types required additional
properties
for configuration.
indexType
Description: Similar to
indexTypes
, but only supports a single index type as a string.Note: If both
indexTypes
andindexType
are present, the latter is ignored.
compressionCodec
Description: An older way to specify compression for indexes.
Recommendation: It's now recommended to specify compression in the forward index config.
Deprecated properties
Description: Before Pinot 0.13, certain indexes were configured using properties within this section.
Migration: Since Pinot 0.13, each index can be configured in a type-safe manner within its dedicated section in the
indexes
object. The documentation for each index type lists the properties that were previously used.Notable Properties:
Text Index Properties:
enableQueryCacheForTextIndex
(used to enable/disable the cache, with values specified as strings, e.g., "true" or "false").Forward Index Properties:
rawIndexWriterVersion
,deriveNumDocsPerChunkForRawIndex
,forwardIndexDisabled
.
It's strongly recommended to migrate from these deprecated options to the new, more structured configuration methods introduced in Pinot 0.13 for better maintainability and compatibility.
Warning:
If removing the forwardIndexDisabled
property above to regenerate the forward index for multi-value (MV) columns note that the following invariants cannot be maintained after regenerating the forward index for a forward index disabled column:
Ordering guarantees of the MV values within a row
If entries within an MV row are duplicated, the duplicates will be lost. Regenerate the segments via your offline jobs and re-push / refresh the data to get back the original MV data with duplicates.
We will work on removing the second invariant in the future.
Real-time table config
The sections below apply to real-time tables only.
segmentsConfig
Indexing config
The streamConfigs
section has been deprecated as of release 0.7.0. See streamConfigMaps
instead.
Tenants
Example
Environment variables override
Pinot allows users to define environment variables in the format of ${ENV_NAME}
or ${ENV_NAME:DEFAULT_VALUE}
as field values in table config.
Pinot instance will override it during runtime.
Brackets are required when defining the environment variable."$ENV_NAME"
is not supported.
Environment variables used without default value in table config have to be available to all Pinot components - Controller, Broker, Server, and Minion. Otherwise, querying/consumption will be affected depending on the service to which these variables are not available.
Below is an example of setting AWS credential as part of table config using environment variable.
Example:
Sample configurations
Offline table
Real-time table
Here's an example table config for a real-time table. All the fields from the offline table config are valid for the real-time table. Additionally, real-time tables use some extra fields.