Broker

You can set broker properties in a configuration file. The file can be provided during startup time as follows -

bin/pinot-admin.sh StartBroker -configFileName /path/to/broker.conf

broker.conf can have the following properties. All properties are defined in this class.

PropertyDefaultDescription

pinot.broker.delayShutdownTimeMs

10 seconds

pinot.broker.enableTableLevelMetrics

true

pinot.broker.query.response.limit

Integer.MAX_VALUE

When config pinot.broker.enable.query.limit.overrideis enabled, reset limit for selection query if it exceeds this value.

pinot.broker.query.log.length

Integer.MAX_VALUE

pinot.broker.query.log.maxRatePerSecond

10000.0

Maximum queries to be logged per second. Queries with exceptions, or take longer than 1 second are always logged.

pinot.broker.timeoutMs

10 seconds

Timeout for Broker Query in Milliseconds

pinot.broker.startup.minResourcePercent

100

Configuration to consider the broker ServiceStatus as being STARTED if the percent of resources (tables) that are ONLINE for this this broker has crossed the threshold percentage of the total number of tables that it is expected to serve

pinot.broker.enable.query.limit.override

false

Configuration to enable Query LIMIT Override to protect Pinot Broker and Server from fetch too many records back.

pinot.broker.client.queryPort

8099

Port to query broker via http (legacy)

pinot.broker.client.access.protocols

Ingress protocols to query broker (http or https or http,https)

pinot.broker.client.access.protocols.http.port

Port to query broker via http

pinot.broker.client.access.protocols.https.port

Port to query broker via https

pinot.broker.netty.enabled

true

Enable unsecured netty connections to pinot-server

pinot.broker.nettytls.enabled

false

Enable secured netty connections to pinot-server

pinot.broker.tls.keystore.path

Path to broker TLS keystore

pinot.broker.tls.keystore.password

keystore password

pinot.broker.tls.truststore.path

Path to broker TLS truststore

pinot.broker.tls.truststore.password

truststore password

pinot.broker.tls.requires_client_auth

false

toggle for requiring TLS client auth

pinot.broker.http.server.thread.pool.corePoolSize

2 * cores

Config for the thread-pool used by pinot-broker's http-server.

pinot.broker.http.server.thread.pool.maxPoolSize

2 * cores

Config for the thread-pool used by pinot-broker's http-server.

pinot.broker.enable.bounded.jersey.threadpool.executor

false

Enable bounded Jersey thread-pool to handle async requests.

pinot.broker.jersey.threadpool.executor.max.pool.size

2 * cores

Config for the bounded Jersey thread-pool to handle async requests.

pinot.broker.jersey.threadpool.executor.core.pool.size

2 * cores

Config for the bounded Jersey thread-pool to handle async requests.

pinot.broker.jersey.threadpool.executor.queue.size

Integer.MAX_VALUE

Config for the bounded Jersey thread-pool to handle async requests.

pinot.broker.max.query.response.size.bytes

Long.MAX_VALUE

Config indicating the maximum serialized response size across all servers for a query. This value is // equally divided across all servers processing the query.

pinot.broker.max.server.response.size.bytes

Long.MAX_VALUE

Config indicating the maximum length of the serialized response per server for a query.

pinot.broker.event.listener.factory.className

org.apache.pinot.spi.eventlistener.query.NoOpBrokerQueryEventListener

Config indicating the event listener class that will be loaded during broker starter and the onQueryCompletion will be called post query completion. It is on the implementation of this method that user can improve query level observability in their system.

pinot.broker.event.listener.request.context.tracked.header.keys

Comma separated string values indicating the request-headers which will be tracked by the event listener class.

pinot.broker.new.segment.expiration.seconds

300 seconds (5 minutes)

Config for broker to consider a new segment as an old segment and start tagging replica-group / instances as unavailable after this time period.

pinot.broker.query.enable.null.handling

false

Config that allows enabling advanced null handling support for all queries by default (see https://docs.pinot.apache.org/developers/advanced/null-value-support#advanced-null-handling-support)

Last updated