Query Options

This document contains all the available query options

Supported Query Options

Key Description Default Behavior

Key	Description	Default Behavior
timeoutMs	Timeout of the query in milliseconds	Use table/broker level timeout
enableNullHandling	Enables advanced null handling. See Null value support for more information.(introduced in 0.11.0)	`false` (disabled)
explainPlanVerbose	Return verbose result for `EXPLAIN` query (introduced in 0.11.0)	`false` (not verbose)
useMultistageEngine	Use multi-stage engine to execute the query (introduced in 0.11.0)	`false` (use single-stage engine)
maxExecutionThreads	Maximum threads to use to execute the query. Useful to limit the resource usage for expensive queries	Half of the CPU cores for non-group-by queries; all CPU cores for group-by queries
numReplicaGroupsToQuery	When replica-group based routing is enabled, use it to query multiple replica-groups (introduced in 0.11.0)	`1` (only query servers within the same replica-group)
minSegmentGroupTrimSize	Minimum groups to keep when trimming groups at the segment level for group-by queries. See	Server level config
minServerGroupTrimSize	Minimum groups to keep when trimming groups at the server level for group-by queries. See	Server level config
skipIndexes	Which indexes to skip usage of (i.e. scan instead), per-column. This is useful for side-by-side comparison/debugging. There can be cases where the use of an index is actually more expensive than performing a scan of the docs which match other filters. One such example could be a low-selectivity inverted index used in conjunction with another highly selective filter. Config can be specified using url parameter format: `skipIndexes='col1=inverted,range&col2=inverted'` Possible index types to skip are: `sorted, range, inverted, H3`. To find out which indexes are used to resolve a given query, use the `EXPLAIN` query.	`null/empty` (use all available indexes)
skipUpsert	For upsert-enabled table, skip the effect of upsert and query all the records. See Stream ingestion with Upsert	`false` (exclude the replaced records)
useStarTree	Useful to debug the star-tree index (introduced in 0.11.0)	`true` (use star-tree if available)
AndScanReordering	See detailed description	disabled
maxRowsInJoin	Configure maximum rows allowed in a join operation. This limit is applied to both the hash table build phase for the join's right input as well as the number of joined rows emitted after matching with the join's left input.	default value read from cluster config `pinot.query.join.max.rows` if not set, the default will be *2^20 (10241024)**
inPredicatePreSorted	(Only apply to STRING columns) Indicates that the values in the IN clause is already sorted, so that Pinot doesn't need to sort them again at query time	`false` (values in IN predicate is not pre-sorted)
inPredicateLookupAlgorithm	(Only apply to STRING columns) The algorithm to use to look up the dictionary ids for the IN clause values. `DIVIDE_BINARY_SEARCH`: Sort the IN clause values and do binary search on both dictionary and IN clause values at same time to reduce the value lookups `SCAN`: Sort the IN clause values and scan both dictionary and IN clause values to get the matching dictionary ids `PLAIN_BINARY_SEARCH`: Do not sort the IN clause values, but directly binary search each IN clause value in the dictionary	`DIVIDE_BINARY_SEARCH`
maxServerResponseSizeBytes	Long value config indicating the maximum length of the serialized response per server for a query.	Overriding priortiy order: 1. QueryOption -> maxServerResponseSizeBytes 2. QueryOption -> maxQueryResponseSizeBytes 3. TableConfig -> maxServerResponseSizeBytes 4. TableConfig -> maxQueryResponseSizeBytes 5. BrokerConfig -> maxServerResponseSizeBytes 6. BrokerConfig -> maxServerResponseSizeBytes
maxQueryResponseSizeBytes	Long value config indicating the maximum serialized response size across all servers for a query. This value is equally divided across all servers processing the query.	Overriding priortiy order: 1. QueryOption -> maxServerResponseSizeBytes 2. QueryOption -> maxQueryResponseSizeBytes 3. TableConfig -> maxServerResponseSizeBytes 4. TableConfig -> maxQueryResponseSizeBytes 5. BrokerConfig -> maxServerResponseSizeBytes 6. BrokerConfig -> maxServerResponseSizeBytes
filteredAggregationsSkipEmptyGroups	This config can be set to `true` to avoid computing all the groups in a group by query with only filtered aggregations (and no non-filtered aggregations). By default, the groups are computed over all the rows returned by the main filter, even if certain rows will never match any of the aggregation filters. This is the standard SQL behavior. However, if the selectivity of the main filter is very high as compared to the selectivity of the aggregation filters, this query option can help provide a big performance boost if the empty groups aren't required. For instance, a query like `SELECT SUM(X) FILTER (WHERE Y = 1) FROM mytable` will compute the groups over all the rows in the table by default since there's no main query filter. Setting this query option to `true` in such cases can massively improve performance if there's an inverted index on column `Y` for instance.	`false` (i.e., all groups are computed by default as per standard SQL)

timeoutMs

Timeout of the query in milliseconds

Use table/broker level timeout

enableNullHandling

Enables advanced null handling. See Null value support for more information.(introduced in 0.11.0)

false (disabled)

explainPlanVerbose

Return verbose result for EXPLAIN query (introduced in 0.11.0)

false (not verbose)

useMultistageEngine

Use multi-stage engine to execute the query (introduced in 0.11.0)

false (use single-stage engine)

maxExecutionThreads

Maximum threads to use to execute the query. Useful to limit the resource usage for expensive queries

Half of the CPU cores for non-group-by queries; all CPU cores for group-by queries

numReplicaGroupsToQuery

When replica-group based routing is enabled, use it to query multiple replica-groups (introduced in 0.11.0)

1 (only query servers within the same replica-group)

minSegmentGroupTrimSize

Minimum groups to keep when trimming groups at the segment level for group-by queries. See

Server level config

minServerGroupTrimSize

Minimum groups to keep when trimming groups at the server level for group-by queries. See

Server level config

skipIndexes

Which indexes to skip usage of (i.e. scan instead), per-column. This is useful for side-by-side comparison/debugging. There can be cases where the use of an index is actually more expensive than performing a scan of the docs which match other filters. One such example could be a low-selectivity inverted index used in conjunction with another highly selective filter.

Config can be specified using url parameter format: skipIndexes='col1=inverted,range&col2=inverted'

Possible index types to skip are: sorted, range, inverted, H3. To find out which indexes are used to resolve a given query, use the EXPLAIN query.

null/empty (use all available indexes)

skipUpsert

For upsert-enabled table, skip the effect of upsert and query all the records. See Stream ingestion with Upsert

false (exclude the replaced records)

useStarTree

Useful to debug the star-tree index (introduced in 0.11.0)

true (use star-tree if available)

AndScanReordering

See detailed description

disabled

maxRowsInJoin

Configure maximum rows allowed in a join operation. This limit is applied to both the hash table build phase for the join's right input as well as the number of joined rows emitted after matching with the join's left input.

default value read from cluster config

pinot.query.join.max.rows

if not set, the default will be

2^20 (1024*1024)

inPredicatePreSorted

(Only apply to STRING columns) Indicates that the values in the IN clause is already sorted, so that Pinot doesn't need to sort them again at query time

false (values in IN predicate is not pre-sorted)

inPredicateLookupAlgorithm

(Only apply to STRING columns) The algorithm to use to look up the dictionary ids for the IN clause values.

DIVIDE_BINARY_SEARCH: Sort the IN clause values and do binary search on both dictionary and IN clause values at same time to reduce the value lookups

SCAN: Sort the IN clause values and scan both dictionary and IN clause values to get the matching dictionary ids
PLAIN_BINARY_SEARCH: Do not sort the IN clause values, but directly binary search each IN clause value in the dictionary

DIVIDE_BINARY_SEARCH

maxServerResponseSizeBytes

Long value config indicating the maximum length of the serialized response per server for a query.

Overriding priortiy order: 1. QueryOption -> maxServerResponseSizeBytes

2. QueryOption -> maxQueryResponseSizeBytes

3. TableConfig -> maxServerResponseSizeBytes

4. TableConfig -> maxQueryResponseSizeBytes

5. BrokerConfig -> maxServerResponseSizeBytes

6. BrokerConfig -> maxServerResponseSizeBytes

maxQueryResponseSizeBytes

Long value config indicating the maximum serialized response size across all servers for a query. This value is equally divided across all servers processing the query.

Overriding priortiy order: 1. QueryOption -> maxServerResponseSizeBytes

2. QueryOption -> maxQueryResponseSizeBytes

3. TableConfig -> maxServerResponseSizeBytes

4. TableConfig -> maxQueryResponseSizeBytes

5. BrokerConfig -> maxServerResponseSizeBytes

6. BrokerConfig -> maxServerResponseSizeBytes

filteredAggregationsSkipEmptyGroups

This config can be set to true to avoid computing all the groups in a group by query with only filtered aggregations (and no non-filtered aggregations). By default, the groups are computed over all the rows returned by the main filter, even if certain rows will never match any of the aggregation filters. This is the standard SQL behavior. However, if the selectivity of the main filter is very high as compared to the selectivity of the aggregation filters, this query option can help provide a big performance boost if the empty groups aren't required. For instance, a query like SELECT SUM(X) FILTER (WHERE Y = 1) FROM mytable will compute the groups over all the rows in the table by default since there's no main query filter. Setting this query option to true in such cases can massively improve performance if there's an inverted index on column Y for instance.

false (i.e., all groups are computed by default as per standard SQL)

Set Query Options

SET statement

After release 0.11.0, query options can be set using the SET statement:

SET key1 = 'value1';
SET key2 = 123;
SELECT * FROM myTable

OPTION keyword (deprecated)

Before release 0.11.0, query options can be appended to the query with the OPTION keyword:

SELECT * FROM myTable OPTION(key1=value1, key2=123)
SELECT * FROM myTable OPTION(key1=value1) OPTION(key2=123)
SELECT * FROM myTable OPTION(timeoutMs=30000)

REST API

Query options can be specified in API using queryOptions as key and ';' separated key-value pairs. Alternatively, we can also use the SET keyword in the sql query.

Using Controller Admin API

curl -X POST 'http://localhost:9000/sql' \
-d '{
  "sql": "SELECT * FROM myTable",
  "trace": false,
  "queryOptions":"key1=value1;key2=123"
}'

Using Broker Query API

curl -X POST 'http://localhost:8099/query/sql' \
-d '{
  "sql": "SELECT * FROM myTable;",
  "trace": false,
  "queryOptions":"key1=value1;key2=123"
}'

PreviousFunnel Analysis NextQuery Quotas

Last updated 1 month ago